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Although  future  applications  for  autonomous  mobile  robots  (AMR)  are  practically 
limitless,  researchers  must  first  address  some  of  the  hurdles  blocking  wide  acceptance  of 
AMR  as  a viable  solution.  This  research  addresses  some  of  these  issues  through  the 
realization  of  Innate  Learning  (IL),  Environmental  Reinforcement  Learning  (ERL),  and 
Autonomous  Evolution  of  sensory  and  actuator  Driver  layers  through  Environmental 
Constraints  (AEDEC).  Innate  Learning  (IL)  is  a learning  mechanism  that  takes 
advantage  of  innate  knowledge  to  improve  and  enhance  learning.  Through  the  use  of 
innate  knowledge  of  its  embodiment  and  its  environment,  IL  provides  a simple 
mechanism  to  autonomously  detect  and  correct  discrete  production  errors  (i.e.,  errors  in 
wiring  of  sensors  and  actuators).  Environmental  Reinforcement  Learning  (ERL)  is  a 
real-time  learning  architecture  for  refining  primitive  behaviors  through  interaction  within 
highly  structured  environments.  ERL  architecture  allows  self-calibration  of  sensors. 
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actuators,  and  primitive  behaviors  by  using  a structured  environment  (i.e.,  an  obstacle 
course)  to  provide  real-time  feedback  on  a robot’s  performance.  Through  the  refinement 
process,  lower  cost  parts  can  be  used  and  damaged  parts  can  be  replaced  without 
affecting  the  rest  of  the  system.  Finally,  AEDEC  learning  architecture  is  the  culmination 
of  the  previous  research,  namely  Innate  Learning  (IL)  and  Environmental  Reinforcement 
Learning  (ERL).  By  incorporating  IL  and  ERL,  AEDEC  asserts  that  sensory  and  actuator 
driver  layers  can  be  autonomously  programmed  from  a simple  set  of  innate  knowledge 
guided  by  static  constraints  from  a highly  structured  environment.  Through  the  use  of 
innate  knowledge  and  a highly  structured  environment,  AEDEC  allows  a robot  to 
autonomously  create  abstractions  (drivers)  of  sensory  information  and  actuation  controls, 
consequently  reducing  the  work  load  of  a human  programmer.  Since  different  type  of 
robots  (walking,  two  wheels,  caterpillar  treads,  etc.)  can  be  trained  in  the  same 
environment,  AEDEC  permits  code  (high  level  behavior)  portability  between  different 


types  of  robots. 


CHAPTER  1 
INTRODUCTION 


Research  in  autonomous  mobile  robots  has  grown  into  a promising  field  for 
research  and  for  commercialization.  We  have  seen  many  examples  in  recent  years  of 
how  autonomous  mobile  robots  have  given  humankind  inspiration  for  the  future  and  a 
glimpse  of  things  to  come.  One  notable  example  was  NASA’s  mission  to  Mars.  The 
video  images  of  the  robot  Sojourner  captivated  not  only  the  United  States  but  also  the 
whole  world.  This  mission  showed  how  robots  could  go  “where  no  man  has  gone 
before.”  However,  what  the  general  population  does  not  realize  is  that  autonomous 
mobile  robots  are  just  coming  into  maturity.  There  are  many  issues  still  left  unexplored. 
This  research  hopes  to  explore  some  of  these  issues  and  lay  out  a foundation  upon  which 
to  build  future  research. 

This  research  addresses  one  of  the  essential  problems  that  have  plagued 
researchers  in  autonomous  mobile  robots,  namely  the  complexity  involved  in 
programming  these  robots.  Every  researcher  in  the  field  of  autonomous  mobile  robots 
has  experienced  first  hand  the  tedious  and  finicky  nature  involved  in  programming 
autonomous  mobile  robots.  Even  for  straightforward  algorithms,  the  physical 
implementation  is  often  tedious  and  long.  The  problem  stems  from  the  requirement  that 
the  programmer  needs  detailed  knowledge  of  the  mechanics  and  the  electronics  of  an 
autonomous  mobile  robot  to  be  able  to  program  it.  Consequently,  programming  of 
autonomous  mobile  robots  is  limited  to  a handful  of  people  with  intimate  knowledge  of 
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the  robot’s  mechanics  and  electronics  or  people  willing  to  gain  this  knowledge.  This 
requirement  is  an  obvious  hindrance  in  autonomous  mobile  robot  programming. 

Traditionally,  the  above  mentioned  problem  has  been  addressed  by  writing  drivers 
(low  level  software  controlling  the  hardware)  for  autonomous  mobile  robots.  Usually, 
the  robot  designer  writes  drivers  for  the  robot’s  actuators  and  sensors.  For  example, 
instead  of  having  to  deal  with  the  output  port  pin  that  is  connected  to  the  direction  control 
of  the  motor  driver  chip,  the  programmer  could  change  the  direction  of  the  motor  just  by 
changing  the  sign  (motor(0,100)  =>  turn  motor  0 at  a speed  of  100;  motor(0,-100)  =>  turn 
motor  0 in  the  opposite  direction  at  a speed  of  100).  Although  this  level  of  abstraction 
greatly  simplifies  the  programming  of  autonomous  mobile  robots,  the  programmer  still 
has  to  figure  out  the  effect  of  each  motor  on  the  robot’s  motion  characteristics.  In  other 
words,  the  programmer  still  needs  to  determine  which  motor  is  motor  0 and  which 
direction  the  motor  will  turn  when  a positive  speed  is  given.  Therefore,  by  having  a 
higher  level  of  abstraction  of  actuators  and  sensors,  the  programming  of  autonomous 
mobile  robots  can  be  further  simplified. 

However,  someone  still  needs  to  develop  and  program  the  drivers  for  each  of  the 
different  autonomous  mobile  robots.  If  a robot  could  learn  to  create  its  own  driver  layers 
for  the  sensors  and  actuators,  the  burden  on  the  programmer  would  be  greatly  reduced. 
Furthermore,  what  if  a robot  could  learn  to  self  configure  its  sensors  and  actuators  (have 
the  robot  learn  which  sensors  are  in  the  front  and  which  are  in  the  back  which  motor  is 
the  left  motor,  which  polarity  is  forward).  This  would  greatly  simplify  the  programming 
requirements  for  a robot. 
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This  research  outlines  a systematic  approach  and  the  supporting  theories  that  will 
facilitate  the  autonomous  configuration  and  development  of  the  sensory  and  actuator 
driver  layers.  The  research  is  separated  into  three  parts:  autonomous  configuration  of 
actuators  and  sensors  through  Innate  Learning  (IL)  [15],  autonomous  primitive  behavior 
refinement  through  Environmental  Reinforcement  Learning  (ERL)  [14],  and  autonomous 
abstraction  of  sensor  information  and  of  actuator  commands  through  environmental 
constraints.  In  essence,  a robot  will  become  aware  of  its  sensory  and  actuator 
embodiments  through  Innate  Learning.  Using  the  knowledge  gained  in  the  first  stage,  a 
robot  will  then  learn  to  refine  its  primitive  behaviors  through  Environmental 
Reinforcement  Learning.  By  incorporating  IL  and  ERL,  Autonomous  Evolution  of 
sensory  and  actuator  Driver  layer  through  Environmental  Constraints  (AEDEC)  asserts 
that  sensory  and  actuator  driver  layers  can  be  autonomously  created  from  a simple  set  of 
innate  knowledge  guided  by  static  constraints  from  a highly  structured  environment. 

This  dissertation  is  organized  as  follows:  Chapter  2 presents  a brief  survey  of 
current  trends  in  autonomous  mobile  robot  learning,  especially  in  the  context  of  robot 
controller  evolution.  Chapter  3 discusses  some  of  the  issues  facing  the  researcher  in  the 
field  of  autonomous  mobile  robotics.  Chapter  4 introduces  Mantaray  and  Talrik  II™, 
autonomous  mobile  robots  used  in  the  experiments.  Chapter  5 describes  Innate  Learning 
[15],  a learning  architecture  which  incorporates  innate  knowledge  in  its  learning  process. 
Chapter  6 describes  Environmental  Reinforcement  Learning  [14],  a learning  architecture 
which  incorporates  a highly  structured  environment  in  its  learning  process.  Both  Chapter 
5 and  Chapter  6 are  adaptations  from  previously  published  papers.  Chapter  7 introduces 
Autonomous  Evolution  of  sensory  and  actuator  Driver  layer  through  Environmental 
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Constraints  (AEDEC).  Chapter  8 presents  sensory  driver  layer  learned  using  AEDEC 
architecture  and  the  supporting  experiments  and  analysis  of  the  sensory  driver  layer. 
Similarly,  Chapter  9 presents  the  actuator  driver  layer  learned  using  AEDEC  architecture 
and  the  supporting  experiments  and  analysis  of  the  actuator  driver  layer.  Chapter  10 
discusses  the  new  programming  concepts  and  the  high  level  behavior  code  portability 
with  in  the  AEDEC  architecture.  In  support  of  AEDEC,  obstacle  avoidance  and  wall 
following  behaviors  were  implemented  and  presented.  Finally,  Chapter  1 1 ends  with  a 
summary  of  the  concepts  and  their  results. 


CHAPTER  2 

SURVEY  OF  CURRENT  RESEARCH  IN 
AUTONOMOUS  MOBILE  ROBOT  LEARNING 

The  classical  approaches  to  robotics  have  been  to  decompose  the  problem  into 
perception,  planning,  and  action  modules.  Recently,  many  researchers  agree  that  the 
decomposition  was  wrong.  Even  Brooks  [5]  now  admits  that  his  subsumption 
architecture  does  not  address  the  following  two  problems. 

1.  Interactions  between  separate  subsystems  are  not  limited  to  directly  visible 
connecting  links  between  them,  but  also  include  interactions  mediated  via  the 
environment. 

2.  As  system  complexity  grows,  the  number  of  potential  interactions  between 
subpart  of  the  system  grows  exponentially.  [27] 

The  subsumption  architecture  [5]  asserts  that  robot  control  should  be  built  layer 
by  layer,  where  each  layer  is  responsible  for  a new  behavior.  It  also  states  that  interaction 
between  layers  should  be  limited  to  inhibition,  suppression,  or  message  passing  from  a 
higher  layer  to  a lower  layer.  In  practice,  the  condition  of  limited  interaction  is 
sometimes  breached  and  despite  efforts  to  minimize  interactions,  the  interactions  become 
unpredictable  and  unmanageable  when  the  number  of  behavior  layers  increases  beyond 
10  layers  [27]. 

The  obvious  alternative  is  to  hide  the  problems  of  interactions  by  incorporating 
learning  to  design  robot  controllers,  rather  than  attempting  to  hand  design  each  new 
behavior  layer.  Unlike  the  hand  coding  method,  a learning  algorithm  does  not  need  to  be 
bothered  by  unanticipated  and  unobservable  interactions  between  the  various  behavior 
layers.  Consequently,  researchers  in  the  field  of  autonomous  mobile  robots  are  using 
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many  different  approaches  when  it  comes  to  learning  in  robotics.  Some  have  benefited 
by  combining  different  approaches  to  form  hybrid  methods,  notably  the  hybrid  approach 
of  combining  genetic  programming  (GP)  with  Neural  Networks  (NN).  This  section 
introduces  selected  research  from  different  types  of  learning  paradigms. 


Although  interest  in  reinforcement  learning  in  the  field  of  machine  learning  has 
rapidly  increased  in  the  past  ten  years  or  so,  its  beginning  is  associated  with  work  in 
statistics,  psychology,  neuroscience,  and  computer  science.  Its  attraction  stems  from  the 
promise  that  a robot  can  be  programmed  though  reward  and  punishment  without 
explicitly  hard  coding  the  behavior.  Therefore,  reinforcement  learning  is  a learning 
process  by  which  a robot  learns  a behavior  though  trial  and  error  interactions  with  the 
environment. 


Figure  2.1  The  standard  reinforcement  learning  model  [31] 

The  two  main  approaches  in  reinforcement  learning  consist  of  either  searching  the 
behavior  space  for  the  one  that  performs  the  “best”  or  estimating  the  consequence  of  a 
particular  action  through  statistical  techniques  and  dynamic  programming  methods.  The 
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standard  reinforcement  learning  model  is  very  simple,  as  shown  in  Figure  2.1.  At  each 
interval,  the  robot  receives  the  input  (/),  and  the  current  states  (5),  and  then  performs  an 
action  (a).  The  execution  of  an  action  changes  the  state  of  the  environment,  which  results 
in  a scalar  reinforcement  signal  (r).  By  choosing  actions  that  increase  the  reinforcement, 
a robot  ultimately  learns  a behavior  through  a trial  and  error  process.  In  reinforcement 
learning,  a robot  must  explicitly  explore  its  environment,  which  can  be  a very  large 
search  space.  Due  to  the  large  search  space  in  most  meaningful  applications, 
reinforcement  learning  usually  require  large  number  of  iterations  to  learn  (converge  to)  a 
goal.  Although  the  basic  concept  of  reinforcement  learning  is  straight  forward,  many 
different  architectures  and  algorithms  have  been  proposed  for  actual  implementation  of 
reinforcement  learning  to  alleviate  the  long  convergence  time. 

2.2  Genetic  Programming 

Genetic  algorithms  (GA),  introduced  by  Holland  [29],  are  optimization  methods, 
which  imitate  the  genetic  processes  of  evolution  such  as  selection,  recombination,  and 
mutation.  GA  starts  with  a population  of  candidates,  each  with  its  genetic  string  of 
parameters  called  chromosomes.  After  each  generation,  the  “fittest”  candidates  survives 
while  the  “weakest”  are  eliminated.  The  new  generation  inherits  genetic  code  from  their 
parent,  usually  receiving  parts  from  two  parents.  In  addition,  some  mutations  are  also 
introduced,  so  they  may  keep  the  system  from  getting  stuck  in  a local  minimum.  By 
increasing  the  mutation  rates,  one  could  increase  the  exploration  of  the  search  space,  but 
if  the  mutation  rate  is  too  high,  then  the  population  loses  the  current  local  optimum  and 
the  search  becomes  random  [21].  Through  the  passing  of  each  generation,  the  initial 
candidates  evolve  into  more  suitable  solutions  to  the  problem.  Although  GA  has  been 
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used  for  search,  optimization,  and  machine  learning  techniques,  GA  was  originally 
intended  for  complex  adaptive  systems.  Evolutionary  robotics  typically  requires 
improvements  in  adaptation  rather  than  optimization. 

John  Koza  [32,  33]  introduced  genetic  programming  in  the  context  of  evolving 
controllers  in  simulation.  GP  manipulates  higher  level  primitive  constructs,  unlike 
genetic  algorithms,  which  manipulate  bit  strings.  Due  to  the  abstraction  in  GP,  one  can 
significantly  reduce  the  search  space  and  thereby  accelerate  the  evolutionary  process. 
However,  the  creation  of  the  primitives  is  more  of  an  art  form  than  a scientific  process. 
Furthermore,  most  of  the  GP  techniques  have  been  implemented  only  in  simulations. 
Only  a small  number  of  researchers  have  taken  their  results  from  the  simulations  and 
transferred  them  onto  physical  robots.  Moreover,  very  few  have  implemented  the 
evolutionary  process  directly  on  physical  robots  in  real  time  [37], 

In  simulation,  Beer  and  Gallagher  [3]  demonstrated  that  neural  networks  could 
produce  coordinated  sensory-motor  behavior  for  two  tasks,  namely,  chemotaxis  and 
locomotion.  They  used  genetic  algorithms  to  develop  continuous  time  recurrent  neural 
network  controllers  for  a six-legged  insect-like  robot. 

Shaping  can  significantly  accelerate  the  learning  process,  since  one  can,  through 
observation,  target  the  learning  specifically  to  the  part  of  the  behavior  that  has  not  been 
learned.  Nevertheless,  the  performance  depends  heavily  on  proper  human  intervention. 
Colombetti  and  Dorigo  [16,  17]  applied  incremental  learning  though  shaping.  In 
simulation,  they  implemented  learning  of  basic  behaviors  and  coordination  of  these  basic 
behaviors  under  different  conditions.  Finally,  the  two  types  of  learning  were  merged. 
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However,  on  a real  robot,  the  experiment  was  much  simpler  when  compared  to  the 
simulation. 

Unlike  the  vast  majority  of  work  which  involves  software  evolution  and  learning, 
Thompson  [50]  showed  that  physical  semiconductor  circuits  (robot  control  hardware)  can 
be  evolved.  Another  rarity  is  the  work  published  by  Floreano  and  Mondada  [22,  23], 
who  have  attempted  to  evolve  robot  controllers  using  GA  entirely  on  a physical  robot  in 
real  time  without  any  human  supervision.  The  task  was  to  navigate  around  a circular 
corridor  with  obstacle  avoidance. 

2.3  Neural  Networks 

Neural  networks  [10,  11,  38,  42,  55]  have  been  implemented  successfully  in 
reactive  navigation,  i.e.,  obstacle  avoidance.  In  neural  networks,  knowledge  is  encoded 
in  the  connection  weights  and  thus  hidden  from  the  human  designer.  Due  to  the  encoded 
nature  of  the  weights  in  neural  networks,  the  knowledge  is  difficult  to  evaluate  and 
refinement  of  the  knowledge  through  human  expertise  is  very  difficult.  Other 
weaknesses  of  neural  networks  are  that  they  suffer  from  long  convergence  time  and  do 
not  lend  themselves  to  incremental  learning.  When  new  knowledge  is  introduced  to  the 
trained  neural  network,  it  requires  a new  training  session. 

2.4  Fuzzy  Logic 

Lotfi  Zadeh  [54]  founded  the  principles  of  fuzzy  logic  (FL)  in  1965.  Fuzzy 
logic’s  appeal  is  in  its  ability  to  represent  and  to  process  imprecise  and  uncertain 
knowledge.  Another  appealing  characteristic  is  the  robustness  of  fuzzy  logic  with  respect 
to  noise  and  variations  of  system  parameters.  Thus,  fuzzy  logic  has  been  used  for  control 
of  autonomous  mobile  robots  [4,  25,  34,  44,  47,  49,  52]. 
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As  in  neural  networks,  fuzzy  logic  has  been  successfully  applied  to  reactive 
navigation.  The  similar  success  of  the  two  approaches  is  expected,  since  they  are  proved 
to  be  theoretically  equivalent  [8].  In  fuzzy  logic,  knowledge  is  expressed  in  terms  of  a 
linguistic  rule  that  a human  expert  can  modify.  When  compared  to  neural  networks,  the 
learning  process  of  a fuzzy  system  is  relatively  fast,  and  fuzzy  systems  allow  for  easy 
integration  of  new  knowledge.  However,  derivation  of  the  fuzzy  rules  is  difficult  and 
unreliable,  especially  in  a complex  environment. 

In  light  of  the  unreliable  methods  of  fuzzy  rule  generation,  researchers  have 
sought  ways  to  automatically  generate  the  fuzzy  rules  in  the  context  of  robot  controllers. 
Most  of  the  work  has  taken  place  in  the  domain  of  simulations  [1,4,  34,  47].  However, 
Castellano  et  al.  [9]  demonstrates  that  fuzzy  rules  can  be  extracted  from  a human  operator 
driving  the  vehicle  in  training  situations,  namely  a wall  following  behavior.  The 
generation  of  the  fuzzy  rules  is  not  from  the  ground  up;  instead,  Castellano  et  al. 
“preprocesses”  the  sensor  into  hierarchical  groups. 

2.5  Hybrid  Approaches 

Since  each  method  presented  above  has  strengths  and  weaknesses,  some 
researchers  have  created  hybrid  approaches  by  integrating  two  or  more  methods. 

One  of  these  hybrid  methods  is  the  so-called  genetic  fuzzy  system  (GFS)  [2,  18, 
19],  of  which  there  are  two  main  approaches.  The  first  method  optimizes  the  parameters 
of  the  preexisting  fuzzy  rules,  while  the  second  method  generates  the  fuzzy  rules  without 
the  benefit  of  prior  knowledge.  Hoffmann  and  Pfister  [28]  demonstrate  the  first  method 
by  developing  the  fuzzy  logic  controller  for  obstacle  avoidance  and  “reaching  an  aiming 
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point.”  The  experiment  was  implemented  first  in  the  simulator  and  then  transferred  to  a 
physical  robot. 


CHAPTER  3 

ISSUES  IN  ROBOT  LEARNING 


The  goal  of  learning  in  robotics  is  to  one  day  replace  the  need  for  human 
programming  of  the  robot  and  ultimately  have  the  robots  autonomously  develop  behavior 
that  would  be  too  complex  or  too  time  consuming  for  humans  to  develop.  Although 
current  research  in  robot  learning  shows  promising  signs  of  the  possibilities  of  this 
ultimate  goal,  no  current  research  has  learned  a behavior  that  is  particularly  difficult  for  a 
human  to  implement.  Many  issues  must  be  addressed  before  this  ultimate  goal  can  be 
reached.  This  section  addresses  some  of  the  issues  faced  by  researchers  in  robot  learning. 

3.1  Issues  in  Robot  Learning  for  a Physical  Robot 

Learning  in  physical  robots  takes  a long  time,  making  it  impractical  for  real  world 
implementation.  For  example,  when  Floreano  and  Mondada  [22]  evolved  a collision  free 
navigation,  65  hours  were  needed  for  the  learning  process.  This  was  just  for  a simple 
task.  Clearly,  a more  complicated  task  would  take  much  longer.  Besides  the  obvious 
problems  of  a long  learning  process,  the  long  learning  time  causes  problems  for  the  robot 
in  terms  of  its  power  source.  Although  some  robots  are  tethered  to  an  external  power 
source,  not  all  platforms  can  be.  This  puts  a severe  constraint  on  the  robot’s  ability  to 
learn  a complex  behavior  without  some  other  intervention.  Furthermore,  the  mechanical 
platform  cannot  withstand  the  wear  and  tear  of  constant  testing  without  maintenance  and 
repair.  This  becomes  even  worse  if  one  factors  in  a more  complex  behavior,  which 
would  require  even  longer  time  for  the  learning  process. 
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3.2  Issues  in  Robot  Learning  in  Simulations 

For  anyone  with  experience  with  simulators  in  the  robotics  domain,  the  difficulty 
of  accurately  simulating  physical  systems  is  painfully  obvious  [6].  As  shown  by  Jakobi 
et  al.  [30],  Brooks  [7],  and  Smithers  [46],  developing  a simulator  is  no  easy  task.  Jakobi 
et  al.  took  extreme  care  to  match  the  noise  levels  in  the  simulator  to  the  real  world.  In 
fact,  they  even  hint  that  the  success  of  their  results  of  implementing  their  simulated 
results  on  a real  robot  might  be  due  to  the  simplicity  of  the  Khepera  robot. 

The  problem  with  simulators  arises  from  the  fact  that  it  is  impossible  to  simulate 
all  details  of  the  real  physical  system.  This  forces  simulators  to  make  abstractions  of  real 
world  properties.  Due  to  this  abstraction,  the  learned  behavior  can  be  the  result  of 
properties  in  the  simulator  that  is  not  present  or  that  is  not  the  same  in  the  real  world. 
Consequently,  the  transfer  of  knowledge  from  a simulator  to  a physical  robot  is  extremely 
hard.  Nolfi  et  al.  [40]  acknowledge  the  problems  of  simulation  and  proposed  that  the 
learning  process  should  be  continued  further  on  a physical  robot  to  alleviate  the 
inadequacies  of  the  simulation. 

In  order  to  develop  a robust  simulator,  one  must  base  the  simulator  on  accurate 
physical  measurements  incorporated  into  the  models  of  the  sensors  and  actuators  [37]. 
This  dependence  on  accurate  measurements  of  the  physical  world  forces  the  simulator  to 
be  designed  for  a particular  system;  consequently,  the  simulator  can  not  be  generalized  to 
any  other  system.  Furthermore,  the  physical  dependence  of  the  simulators  has  made  the 
development  of  the  simulators  a very  time  consuming  process.  The  complexity  of  the 
simulator  design  has  forced  most  researchers  to  make  coarse  abstractions  of  the  physical 
world,  deferring  the  final  design  until  after  the  evaluation  of  a physical  system.  As  robots 
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become  more  complex  in  their  sensor  and  actuator  capabilities  and  as  the  robots  given 
tasks  become  more  complex,  the  burden  on  the  simulator  will  increase.  Consequently, 
one  must  question  whether  it  is  worthwhile  to  develop  a simulator  that  is  so  time 
consuming  and  so  specialized. 

3.3  Issues  in  Evaluation  of  Robot  Learning 

The  robotics  community  has  long  faced  the  problem  of  repeatability,  and 
consequently  evaluation,  of  an  experiment  or  research.  The  lack  of  repeatability  results 
from  the  same  source  that  makes  simulation  design  so  complex  and  time  consuming.  The 
problem  is  the  presence  of  large  noise  in  the  sensors,  actuators,  and  the  environment. 
When  such  variance  exists  among  multiple  trials,  it  is  difficult  to  establish  a standard 
performance  against  which  one  can  measure  the  performance  of  a robotic  system. 
Although  humans  can  make  reasonable  evaluations  of  performance,  quantitative  analysis 


is  rare  [16,  36]. 


CHAPTER  4 

AUTONOMOUS  MOBILE  ROBOTS 

Autonomous  mobile  robots  presented  in  this  section  were  designed  and  built  by 
members  of  the  Machine  Intelligence  Laboratory  (MIL)  at  the  University  of  Florida, 
under  the  guidance  of  Dr.  A.  A.  Arroyo  (MIL  Director).  There  are  many  other  ongoing 
projects  in  MIL  and  details  of  these  projects  are  available  at  http:/www. mil. ufl.edu/.  It 
should  be  noted  that  whenever  possible,  the  robot  platforms  are  built  out  of  “off  the 
shelf’  components  to  keep  the  cost  down.  However,  one  should  not  conclude  that  these 
robots  are  limited  in  their  use.  In  fact,  they  are  very  versatile  and  are  well  suited  for 
many  applications,  as  demonstrated  in  the  various  experiments  in  the  following  sections. 

4.1  Talrik  II™ 

Talrik  II™,  shown  in  Figure  4.1,  fits  into  a right  circular  cylinder  ten  inches  in 
diameter  and  ten  inches  in  height.  The  Mekatronix™  MRC11  robot  controller  provides 
the  Talrik  II™  with  an  MC68HC11  microcontroller,  64  Kbytes  of  memory,  and  memory 
mapped  I/O.  The  MRSX01  board  has  recharging  circuitry  with  diode  protection.  Twelve 
IR  emitters  illuminate  Talrik  II™’ s environment  with  invisible  infrared  light.  The  Talrik 
II™ ’s  sensor  suite  includes  12  IR  detectors,  10  bumper  switches,  6 CdS  photo  resistors, 
battery  voltage  level  detect,  and  recharge  current  detect. 
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Figure  4. 1 Talrik  II™ 

4.2  Mantaray 

Mantaray  [13],  shown  in  Figure  4.2,  was  developed  at  the  Machine  Intelligence 
Laboratory  in  1995  as  a general  platform  for  autonomous  mobile  robot  experiments.  It  is 
a seven-inch  by  seven  and  a half-inch,  two  wheeled  mobile  agent  with  a skid  caster  for 
support.  It  uses  five  IR  sensors:  three  in  the  front  and  two  in  the  back.  A 68HC1 1 8-Bit 
Microcontroller,  clocked  at  2 MHz  with  32K  SRAM  memory,  controls  Mantaray.  A 
74HC138  address  decoder  supplies  eight  8-bit  memory  mapped  I/O  addresses.  One  of 
the  memory  mapped  I/O  ports  is  used  to  control  the  independent  operation  of  the  IR 
LED’s  modulated  at  40  kHz.  Using  the  A/D  capability  of  the  68HC11  8-bit 
microcontroller,  the  Mantaray  comes  equipped  with  eight  A/D  ports  (E0  - E7).  These 
A/D  ports  can  be  used  to  collect  sensory  inputs  from  sensors  with  voltage  level  outputs. 
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The  UDN2993B  dual  H-bridge  motor  driver  can  drive  two  DC  motors  which  in  turn  drive 
the  two  wheels  of  the  robot  platform.  Later,  a voltage  divider  was  added  in  parallel  to  the 
battery  voltage  with  an  analog  tap,  thus  allowing  the  agent  to  sense  the  battery  voltage. 


(c)  Bottom  View  (d)  Strabismic  IR  sensor  pair 


Figure  4.2  Mantaray  with  Strabismic  sensory  array. 


CHAPTER  5 
INNATE  LEARNING 


Animals  are  bom  with  certain  innate  (preprogrammed)  knowledge  about 
themselves  and  about  their  environment.  These  simple  “rules  of  life”  help  them  to 
survive  their  harsh  environment  and  evade  predators.  For  example,  when  baby  sea  turtles 
hatch  from  their  eggshells,  they  instinctively  dig  themselves  out  of  the  sand  and  head 
toward  the  safety  of  the  sea.  Furthermore,  these  instinctive  or  innate  behaviors  can  also 
be  found  in  other  animals.  If  animals  have  certain  innate  rules  that  help  them  start  their 
lives  off  successfully,  shouldn’t  autonomous  agents  also  have  some  innate  rules  to  start 
off  their  existence?  Innate  knowledge  can  not  only  help  autonomous  agents  to  survive 
the  complex  real  world,  but  it  also  opens  up  new  learning  mechanisms  that  take 
advantage  of  the  innate  knowledge  to  simplify  the  learning  process. 

This  chapter  presents  Innate  Learning  [15]  (IL),  a learning  mechanism  that  takes 
advantage  of  innate  knowledge  to  improve  and  enhance  learning.  Through  the  realization 
of  two  learning  tasks,  it  will  be  shown  that  IL  is  a powerful  tool  in  learning.  Using 
simple  innate  knowledge  and  expectations,  a robot  will  be  able  to  autonomously 
configure  its  motors  and  its  sensors.  In  short,  this  chapter  presents  Innate  Learning  (IL) 
through  the  realization  of  two  IL  algorithms  and  demonstrates  how  IL  can  be  used  to 
simplify  the  learning  process. 
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5.1  Introduction 

Much  of  the  research  in  learning  in  autonomous  mobile  agents  has  been  focused 
on  complicated  tasks,  such  as  path  planning,  task  optimization,  etc.  [35,  41,  43,  44],  But 
learning  can  be  performed  at  a much  lower  level,  namely  at  the  level  of  the  sensors  and 
actuators.  Normally,  a programmer  needs  to  understand  the  hardware  wiring  of  the 
agent,  if  he  is  to  successfully  program  it.  What  if  an  agent  had  the  ability  to  recognize  a 
sensor  or  an  actuator  through  its  interaction  with  the  environment?  Then,  an  agent  would 
be  able  to  self  configure  (mapping  proper  sensors  and  actuators  to  their  proper  ports)  its 
sensors  and  actuators,  eliminating  the  requirement  that  the  programmer  understand  the 
hardware  wiring  of  an  agent. 

The  concept  of  innate  knowledge  is  not  new,  at  least  to  researchers  in  other 
disciplines.  It  has  explained  many  phenomena  in  animal  behavior  that  at  first  glance  defy 
logical  explanations.  Detailed  discussions,  both  scientific  and  philosophical,  can  be 
found  in  Tipton  [51]  and  Stich  [48].  They  discuss  how  animals  are  bom  with  certain 
innate  knowledge  and  how  this  innate  knowledge  helps  in  the  formation  of  a new 
behaviors  and  acquiring  new  knowledge.  Many  of  these  researchers  and  philosophers 
assert  that  innate  knowledge  is  innate  because  they  are  essential  for  the  survival  of  the 
animal.  With  such  assertions  being  made  by  distinguished  researchers  in  other  fields, 
robotisists  should  seriously  consider  what  role  innate  knowledge  can  play  in  robot 
learning.  We  are  after  all  trying  to  create  robots  that  can  mimic  insects,  animals,  and 
ultimately  humans. 

Two  different  autonomous  mobile  agents,  Talrik  II™  and  Mantaray  (discussed  in 
more  detail  in  Chapter  4)  were  used  to  realize  the  experiment.  Structurally,  the  two 
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robots  have  different  shapes  and  different  sensors.  Mantaray  has  three  IR  sensors  in  a 
Strabismic  layout  (crossed  eyed),  while  Talrik  II™  has  five  forward  looking  IR  sensors 
pointing  radially  out  from  the  center  and  one  IR  sensor  in  the  back.  Both  agents  have  a 
pair  of  motor  actuators  used  for  the  movement  of  the  agents. 

First,  a functional  obstacle  avoidance  behavior  was  implemented  on  each  agent, 
and  then  the  sensor  and  motor  connections  were  randomly  reconnected  at  the  hardware 
level.  This  random  reconnection  results  in  sensors  being  connected  to  the  wrong  ports, 
where  a sensor  data  from  the  left  sensor  might  actually  be  a sensor  data  from  the  right 
sensor.  Furthermore,  random  connection  of  the  actuators  could  result  in  the  motors  being 
connected  with  the  wrong  polarity  (where  “forward”  command  causes  the  motor  to  turn 
“backward”),  or  even  having  the  left  and  right  motor  switched.  Note  that  random 
connections  are  among  similar  types  of  sensor  and  actuators  (sensors  mixed  with  other 
sensors  and  actuators  mixed  with  other  actuators). 

After  the  random  connections  of  the  sensors  and  actuators  at  the  hardware  level, 
the  Innate  Learning  algorithms  reassign  the  sensors  and  actuators  to  their  proper  port  by 
properly  mapping  them  at  the  software  level.  Using  simple,  innate  knowledge  about  the 
environment  (i.e.,  the  environment  is  a simple  enclosed  area)  and  only  the  sensor  suite 
described  above,  the  agents  are  programmed  to  interact  with  the  environment.  Through 
this  interaction,  the  agents  learn  in  which  direction  the  motor  are  connected  and  which 
sensor  represents  the  left  sensor.  In  essence,  the  autonomous  mobile  agent  will  learn  how 
to  use  its  sensors  and  actuators  using  simple  innate  knowledge  about  its  environment  and 
its  physical  embodiment. 
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When  an  autonomous  mobile  robot  can  learn  to  configure  its  sensors  and 
actuators  autonomously,  it  has  many  possible  applications.  For  example,  a programmer 
will  not  need  a detailed  understanding  of  how  the  hardware  works  in  order  to  program  the 
robot.  All  the  programmer  needs  to  know  is  that  there  is  a left  sensor,  right  sensor,  right 
motor,  etc.  More  importantly,  Innate  Learning  introduces  the  concept  of  innate 
knowledge,  which  helps  in  the  learning  process  of  the  autonomous  mobile  agents. 
Furthermore,  this  research  can  lead  to  better  understanding  of  how  learning  can  be 
implemented  on  autonomous  mobile  robots,  since  it  parallels  what  researchers  in  other 
fields  have  observed  in  their  animal  and  human  studies. 

5.2  Innate  Learning  Algorithm 

Two  algorithms,  motor  configuration  and  sensor  configuration,  were  realized  in 
order  to  demonstrate  Innate  Learning.  Both  used  innate  knowledge  about  an  agent’s 
physical  embodiment  and  about  the  environment  in  order  to  gain  new  knowledge  about 
the  proper  configuration  of  their  sensors  and  actuators. 

5.2.1  Assumptions  about  the  Highly  Structured  Environment 

Certain  assumptions  about  the  environment  were  made  to  simplify  the  learning 
process.  First,  the  environment  must  have  enough  open  space,  where  the  agent  can 
initially  be  placed  without  any  of  its  proximity  sensors  detecting  any  obstacles  near  by. 
The  open  space  allows  the  agent  to  calibrate  its  proximity  sensors  to  its  environment  and 
gives  the  agent  enough  room  to  perform  its  Innate  Learning  behaviors.  Second,  the 
environment  should  have  a “simple”  boundary.  The  shape  of  the  boundary  can  be  just 
about  any  shape  as  long  as  the  boundary  does  not  have  jagged  or  open  sections.  For 
example,  the  boundary  can  be  square,  rectangular,  circular,  triangular,  etc.  In  other 
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words,  the  boundary  of  the  environment  should  be  made  up  of  a simple  collection  of 
straight  walls.  The  third  and  final  assumption  is  that  the  enclosed  environment  should  be 
free  of  obstacles.  The  last  two  assumptions  allow  the  agent  to  have  simple 
“expectations”  about  the  outcome  of  its  interaction  with  the  environment.  In  this  simple 
environment,  the  agent  can  expect  to  encounter  an  open  space,  a wall,  or  a corner.  In 
essence,  the  agent  must  be  able  to  distinguish  between  the  different  situations  that  it 
encounters  within  the  environment. 

5.2.2  Assumptions  about  Autonomous  Mobile  Robots 

The  main  assumption  made  about  the  robot  is  that  it  must  have  some  “awareness” 
of  its  front  side  and  of  its  backside.  This  “awareness”  occurs  naturally  in  animals  since 
the  eyes  are  usually  located  in  the  front  of  the  body.  Similarly,  most  autonomous  mobile 
robots  are  asymmetrical  in  their  physical  construction.  Since  robots  spend  majority  of 
their  time  going  forward,  most  of  the  sensors  are  concentrated  on  the  front.  Due  to  the 
sensory  difference  between  the  front  side  and  the  backside,  the  robot  can  easily 
distinguish  whether  the  wall  is  to  the  front  or  to  the  back.  In  fact,  without  having  any 
specific  knowledge  about  which  sensor  is  connected  to  which  side  of  the  robot,  the  robot 
can  still  differentiate  whether  the  wall  is  in  the  front  or  back.  For  example,  more 
proximity  sensors  will  detect  an  obstacle  if  the  wall  is  in  front  of  the  robot  than  if  the  wall 
is  in  the  back.  By  taking  advantage  of  this  sensory  asymmetry,  a robot  can  easily  become 
aware  of  its  front  side  and  its  backside.  The  second  assumption  is  that  the  agent  knows 
which  sensors  it  physically  has  but  does  not  know  which  sensor  is  connected  to  which 
port.  For  example,  the  agent  knows  that  it  has  three  sensors  in  the  front  (front  left,  center, 
and  front  right,  similar  to  an  animal’s  eyes  and  nose)  and  one  sensor  in  the  back.  Finally, 
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the  agent’s  movement  is  controlled  by  two  motors  and  has  the  ability  to  turn  in  place  or  at 
least  in  a small  circular  path  respect  to  the  enclosed  environment. 

5.2.3  Assumptions  about  Innate  Knowledge 

The  line  between  what  is  and  is  not  innate  knowledge  is  gray  rather  than  being  a 
clear  black  and  white  line.  If  one  programs  all  the  knowledge  needed  by  the  agent  then 
an  agent  has  nothing  to  learn,  since  it  is  created  “smart”.  Furthermore,  too  many  innate 
knowledge  or  innate  knowledge  that  is  too  specific  will  make  an  agent  too  rigid  and 
hinder  the  learning  process.  Innate  knowledge  comes  in  many  different  forms  as 
observed  by  behavioral  biologist  during  their  study  of  different  species.  Innate 
knowledge  in  animals  is  a culmination  of  millions  of  years  of  evolution  and  fine-tuning. 
Consequently,  this  presentation  of  IL  does  not  pretend  to  unearth  the  deep  secrets  of 
nature;  instead,  effort  is  made  to  show  the  viability  and  the  power  of  IL  in  the  field  of 
autonomous  mobile  robots. 

With  this  in  mind,  the  detailed  explanations  of  innate  knowledge  coded  into  the 
agents  are  discussed.  The  first  innate  knowledge  coded  into  the  agents  is  that  they  will 
start  their  “lives”  in  a closed  environment.  The  agents  do  not  have  any  detailed 
knowledge  of  the  environment.  In  fact,  all  the  agents  know  is  that  when  they  go  forward, 
they  will  bump  into  something  on  the  front.  The  first  innate  knowledge  is  a simple 
action-reaction  relationship,  but  it  depends  on  the  specific  structure  of  the  environment. 
The  specific  nature  of  the  environment  seems  very  restrictive,  but  it  is  a common 
occurrence  of  nature.  For  example,  animals  tend  to  be  very  selective  about  where  they 
lay  their  young.  Most  try  to  find  a suitable  environment  to  lay  their  young,  since  the 
environment  plays  a significant  role  in  the  survival  of  the  young. 
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The  second  innate  knowledge  is  about  their  embodiment.  It  states  that  the  agents 
can  somehow  differentiate  between  their  front  side  and  backside.  In  the  experiments 
presented  in  this  chapter,  the  agents  are  told  that  their  front  side  is  biased  by  heavily 
populated  sensor  suite.  Although  the  agents  initially  do  not  have  any  idea  about  which 
sensor  is  connected  to  which  port,  it  does  receive  stimuli  (sensory  data)  from  all  of  the 
sensors.  By  observing  the  stimuli  during  it  interaction  with  the  environment,  the  agents 
can  detect  whether  the  wall  of  the  environment  is  to  the  front  side  or  to  the  backside. 
More  sensors  will  be  stimulated  if  the  wall  is  to  the  front  than  if  the  wall  is  to  the  back. 
Again,  the  second  innate  knowledge  is  common  in  animals.  Most  animal  have  a 
concentration  of  sensors  on  the  front  side  of  their  body.  For  example,  eyes,  ears,  nose, 
and  tongue  are  usually  located  on  the  front  side  of  an  animal. 

The  third  and  last  innate  knowledge  is  little  more  specific  than  the  first  two.  It 
states  that  the  agents  have  innate  knowledge  about  approximately  where  each  of  the 
sensors  is  located  on  their  respective  bodies.  Specifically,  the  agents  know  that  there 
exists  a big  gap  between  the  leftmost  sensor  and  the  rightmost  sensor  as  one  circle  around 
the  agents.  Furthermore,  the  agents  know  how  many  sensors  are  in  the  front  and  how 
many  are  in  the  back.  Although  this  last  innate  knowledge  was  used,  it  was  only  used  to 
show  the  versatility  of  innate  knowledge.  In  fact,  configuration  of  the  sensor  should  be 
integrated  more  tightly  with  the  configuration  of  the  actuators.  Details  will  be  discussed 
in  later  chapters. 

5.2.4  Autonomous  Calibration  of  the  Sensors 

Although  the  calibration  of  the  sensors  could  have  been  hard  coded  into  the 
learning  programs,  self-calibration  of  the  sensors  was  used,  due  to  the  advantages 
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provided  by  self-calibration.  First,  the  programmer  does  not  have  to  waste  time 
characterizing  each  of  the  sensors.  Unless  you  are  using  high  quality  (expensive) 
sensors,  one  sensor  will  be  slightly  different  from  the  next.  Ignoring  these  differences 
can  cause  unwanted  side  effects  that  could  severely  hinder  the  performance  of  the  agent. 
These  differences  must  be  taken  into  account  for  reliable  performance  of  the  agent. 
Second,  self-calibration  allows  the  agent  to  adapt  to  changing  ambient  conditions  in  the 
environment.  This  adaptability  is  critical  if  you  have  any  behavior  that  depends  heavily 
on  threshold  values  of  the  sensors. 

5.2.5  Autonomous  Configuration  of  Its  Actuators 

If  the  two  motors  are  arbitrarily  connected,  there  are  eight  possible  combinations, 
of  which  only  one  combination  is  correct.  The  permutations  are  the  possibility  of  left  and 
right  motors  being  switched  (two  possibilities),  the  possibility  of  left  motor  direction 
being  wrong  (two  possibilities),  and  the  possibility  of  right  motor  direction  being  wrong 
(two  possibilities).  This  results  in  eight  possible  combinations.  However,  the 
permutation  resulting  from  the  possibility  of  the  motors  being  switched  can  be  ignored 
for  the  moment,  since  the  movement  of  the  agent  is  not  effected  in  terms  of  moving 
forward,  moving  backward,  or  turning  in  place.  The  consequences  of  having  the  left  and 
right  motors  switched  will  be  discussed  further  in  Section  5.2.6.  Even  if  the  motors  are 
switched,  the  agents  will  still  move  forward  if  the  directions  of  the  motors  are  correct.  As 
shown  in  Table  5.1,  the  only  effect  the  switch  of  the  motors  has  is  on  the  turning 
direction  of  the  agents.  In  fact,  the  only  consequence  of  the  motors  being  switched  is  that 
the  agent  turns  right  when  it  wants  to  turn  left. 
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A simple  but  effective  Innate  Learning  algorithm  was  developed  for  recognizing 
and  properly  categorizing  the  direction  of  the  actuators  used  for  movement  of  the  agents. 
The  IL  algorithm  takes  advantage  of  the  fact  that  there  is  only  four  possible  combination 
of  motor  direction  as  shown  in  Table  5.1.  Furthermore,  if  the  agents  assume  that  the 
actuators  are  configured  correctly  and  go  forward,  there  are  three  possible  outcomes  from 
its  behavior. 

1)  Move  forward  and  detect  a wall  in  the  front 

2)  Move  backward  and  detect  a wall  in  the  back 

3)  Turn  in  place  and  detect  no  wall. 

Table  5.1  Four  possible  combinations  of  the  motor  directions. 


Left  Motor  Direction 

Right  Motor  Direction 

Result  of  Going  Forward 

Case  1 

CORRECT 

CORRECT 

MOVE  FORWARD 

Case  2 

CORRECT 

BACKWARDS 

TURN  IN  PLACE  (right) 

Case  3 

BACKWARDS 

CORRECT 

TURN  IN  PLACE  (left) 

Case  4 

BACKWARDS 

BACKWARDS 

MOVE  BACKWARD 

The  basic  premise  of  the  Innate  Learning  algorithm  for  motor  configuration  is 
that  the  agents  assume  that  the  current  motor  configuration  is  correct  and  tries  to  go 
forward.  This  action  by  the  agents  will  result  in  two  possible  outcomes.  The  agents  will 
either  detect  the  wall  to  the  front  or  they  won’t.  As  shown  in  Table  5.1,  the  only  situation 
that  results  in  the  agents  detecting  a wall  to  the  front  will  be  when  they  are  moving 
forward.  If  one  turns  in  place,  it  will  never  get  to  the  edge  of  the  environment  to  detect 
any  wall.  If  one  goes  backwards,  then  it  will  detect  a wall  to  the  back  but  not  the  front. 
An  important  point  to  realize  about  this  algorithm  is  that  incorrect  actions  always  leave 
the  front  of  the  agent  free  of  detection.  This  is  a very  important  point  because  there  is  no 
need  to  replace  the  agent  in  the  center  of  the  environment  after  every  wrong  action. 
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By  systematically  trying  different  combinations  of  motor  directions,  the  agents 
will  ultimately  arrive  at  a correct  configuration.  Since  Talrik  II™  has  five  sensors  in  the 
front  and  only  one  in  the  back,  it  can  easily  distinguish  the  front  from  the  back  simply  by 
looking  at  the  number  of  sensors  detecting  an  obstacle.  On  the  other  hand,  Mantaray 
only  has  three  sensors  in  the  front  and  none  in  the  back.  This  means  that  Mantaray  can 
only  distinguish  between  no  detection  (turn  in  place  and  move  backward)  and  front  wall 
detection.  If  Mantaray  had  two  incorrect  directions,  then  it  would  go  backwards  and  hit 
the  wall  in  the  back.  This  in  turn  would  lead  to  detection  by  its  front  sensors  during  a 
turn  in  place  maneuver.  However,  the  IL  algorithm  prevents  this  scenario  from 
occurring,  by  guaranteeing  that  Mantaray  would  go  forward  after  a backward  movement. 
If  Mantaray  started  out  with  correct  motor  directions,  going  backward  is  avoided 
altogether.  If  both  motor  directions  were  initially  wrong,  then  by  changing  both  motor 
directions  (following  the  procedure  outlined  in  Figure  5.1)  would  lead  to  forward 
movement.  If  only  one  of  the  motor  directions  was  wrong,  then  Mantaray  would  turn  in 
place  in  one  direction  and  after  both  motor  directions  are  changed  it  would  turn  in  place 
in  the  other  direction.  After  changing  both  motor  directions,  Figure  5.1  states  that  only 
one  motor  should  change.  If  the  motor  direction  that  was  changed  was  the  one  in  the 
wrong  direction,  then  Mantaray  would  go  forward.  Otherwise,  it  would  go  backwards. 
But  the  final  corrections  to  the  motors  are  again  to  change  both  motor  directions,  which 
would  guarantee  that  Mantaray  would  go  forward  after  a backward  movement. 
Therefore,  regardless  of  the  state  of  motor  directions,  algorithm  shown  in  Figure  5.1 
guarantees  that  an  agent  will  ultimately  configure  the  motors  in  the  right  direction  and 


detect  the  wall  in  the  front. 
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Change  both  motor  directions 
Go  forward 


Change  left  motor  direction 
Go  forward 


c 


END 


3 


-Yes — p. 


Figure  5.1  Innate  Learning  algorithm  for  motor  configuration. 


5.2.6  Autonomous  Configuration  of  Its  Sensors 

One  way  the  agents  can  configure  their  sensors  is  to  take  advantage  of  the 
interactions  between  the  sensors  and  the  environment  coupled  with  the  knowledge  of  the 
physical  location  of  the  sensors.  Due  to  the  random  connections  of  the  sensors,  the 
agents  do  not  know  which  sensor  is  the  “left”  sensor,  but  the  agents  know  that  one  of  the 
sensors  is  the  “left”  sensor.  This  knowledge  of  the  sensor  location  is  part  of  the  agents’ 
innate  knowledge.  If  the  agents  detect  a wall  and  start  to  turn  in  place,  different  sensors 
will  detect  their  maximum  value  at  different  times.  In  fact,  the  pattern  created  by 
recording  the  times,  when  the  maximum  readings  occurred,  should  be  the  same  as  the 
physical  order  of  the  sensors.  Furthermore,  the  time  differences  (GAP  time)  between  the 
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maximum  sensor  readings  should  also  correlate  closely  to  the  physical  distance  between 
the  sensors,  as  shown  in  Figure  5.2.  By  matching  the  time  pattern  created  by  the 
maximum  sensor  readings  to  the  physical  pattern  of  the  sensors,  the  agent  can  easily 
assign  the  correct  sensor  to  the  appropriate  sensor  port.  For  example  in  Figure  5.2,  the 
GAP  time  between  the  three  front  sensors  are  much  smaller  than  the  GAP  time  detected 
between  the  front  sensor  and  the  back  sensor.  Therefore,  the  agents  can  use  the  GAP 
time  between  the  maximum  sensor  readings  to  build  a GAP  time  pattern  of  the  sensors. 

GAP  = 0.2  sec  GAP  = 0.2  sec 

MAX  = 2.7  sec 


Figure  5.2  Example  of  the  sensor  pattern  created  for  maximum  readings. 

The  Innate  Learning  algorithm,  shown  in  Figure  5.3,  creates  and  compares 
multiple  GAP  time  patterns  to  give  robustness  to  the  learning  process.  The  first  pattern  is 
created  from  the  times  recorded  for  the  maximum  readings  of  each  sensor.  The  second 
pattern  is  created  from  the  times  recorded  when  each  of  the  sensors  detects  an  obstacle 
for  the  first  time.  Finally,  the  third  pattern  is  created  from  the  times  recorded  when  each 
of  the  sensors  no  longer  detects  an  obstacle  for  the  first  time.  The  raw  GAP  time  patterns 
are  processed  by  sorting  the  sensors  by  their  respective  times  and  locating  the 
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“characteristic”  (distinctive)  gap.  The  GAP  time  pattern  created  from  the  maximum 
sensor  readings  is  used  to  determine  when  the  data  collection  process  is  over.  The  other 
two  GAP  time  patterns  are  processed  and  compared  against  each  other  to  see  if  there  are 
any  discrepancies.  If  there  are  any  discrepancies,  the  learning  process  is  repeated  until 
the  discrepancies  are  removed. 


Figure  5.3  Innate  Learning  algorithm  for  sensor  configuration 

Previously  in  Section  5.2.5,  the  possible  switch  of  left  and  right  motor  was 
ignored  in  the  motor  configuration  algorithm.  It  will  now  be  addressed  by  showing  how 
the  sensor  configuration  algorithm  takes  care  of  the  motor  switch.  In  the  case  where  the 
motors  are  in  their  proper  ports,  the  sensor  configuration  algorithm  configures  the  sensors 
to  their  proper  ports.  However,  in  the  case  where  the  left  and  right  motors  are  switched, 
the  “turn  right”  behavior  in  the  sensor  configuration  algorithm  causes  the  agents  to  “turn 
left.”  The  consequence  of  the  opposite  turn  is  that  the  sensors  are  also  configured  to  the 
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opposite  side  of  the  agents.  If  the  agent  truly  turns  right,  the  sensors  will  detect  the  wall 
in  a clockwise  order,  while  if  the  agent  turns  left,  the  sensors  will  detect  the  wall  in  a 
counter  clockwise  order.  Therefore,  switching  the  motors  causes  the  mirroring  effect  of 
the  sensors,  but  it  is  automatically  compensated  by  the  mirroring  of  the  motors.  In 
essence,  the  agent’s  left  becomes  right  and  the  agent’s  right  becomes  left.  Consequently, 
when  the  agent  detects  some  thing  to  the  right  (actually  agent  detects  something  to  the 
left),  the  agent  turns  left  (actually  turns  right)  to  avoid  the  obstacle.  Although  the  agents’ 
sense  of  left  and  right  might  be  different  from  the  programmer’s,  the  agents  properly 
correlate  the  proper  motors  to  the  appropriate  sensors. 

5.3  Experiment 

Innate  Learning  was  demonstrated  through  realization  of  two  experimental 
results.  The  first  experiment  consisted  of  IL  algorithms  properly  configuring  the  motors’ 
polarity,  even  after  random  hardware  level  reconnections.  Although  there  are  eight 
possible  combinations  of  which  seven  are  incorrect,  possible  combinations  were  reduced 
to  only  four,  using  a special  relationship  between  the  motors  and  the  sensors  (refer  to 
Section  5.2.6  for  details).  The  second  experiment  finished  the  job  by  properly 
configuring  the  sensors.  This  part  of  the  experiment  involved  the  agents  properly 
categorizing  the  sensors  to  their  proper  ports.  For  example,  the  agents  had  to  distinguish 
the  left  sensor  from  the  right  sensor.  It  should  be  noted  that,  the  left  and  right 
categorization  is  respect  to  the  agents’  definition  and  not  the  human  programmer. 
However,  the  difference  in  definition  becomes  inconsequential  due  to  the  mirroring  of 
both  the  motors  and  the  sensors  (refer  to  Section  5.2.6  for  details). 
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5.3.1  Autonomous  Mobile  Robots 

Two  autonomous  mobile  robots,  Talrik  II™  and  Mantaray,  were  used  for  the 
experiments.  Chapter  4 has  a detailed  discussion  of  the  autonomous  mobile  robots. 
Mobile  platforms  with  differing  characteristics  were  purposely  chosen  to  show  the 
versatility  of  the  IL  algorithms.  Mantaray  has  minimal  amount  of  sensors  for  practical 
obstacle  avoidance  while  Talrik  II™  has  a relatively  a richer  sensor  suite.  Mantaray  has 
three  infra-red  proximity  sensors  in  the  front  while  Talrik  II™  has  five  infra-red 
proximity  sensors  in  the  front  and  one  in  the  back,  bump  sensors  all  around  the  body,  and 
a set  of  CDS  (light  sensitive)  sensor.  Furthermore,  Mantaray  uses  Strabismic  or  crossed 
eyed  sensor  layout  (shown  in  Figure  4.2),  whereas  Talrik  II™  uses  sensors  that  are 
pointing  radially  out. 

5.3.2  Environment 

The  learning  environment  consisted  of  four  white  walls,  each  10  feet  long  and 
four  inches  high.  The  four  walls  were  put  together  to  form  a square  boundary  with  an 
open  area  of  10  feet  by  10  feet.  Except  for  the  agent  involved,  the  open  area  was  devoid 
of  any  other  obstacles.  This  simple  environment  satisfied  the  requirement  stated  in 
Section  5.2.1. 

5.3.3  Result 

With  the  sensors  connected  to  their  proper  ports,  both  agents  were  initially 
programmed  with  a simple  obstacle  avoidance  program,  which  used  all  of  their  respective 
sensors.  This  obstacle  avoidance  program  was  used  as  a test  for  the  Innate  Learning 
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algorithms.  If  the  IL  algorithms  configured  the  sensors  correctly,  the  obstacle  avoidance 
program  would  work;  otherwise,  it  would  not. 

After  randomly  connecting  the  motors  and  the  sensors,  it  was  verified  that  the 
obstacle  avoidance  programs  performed  very  poorly  with  the  new  random  connections  of 
its  sensors  and  their  motors.  Then  both  Talrik  II™  and  Mantaray  executed  the  learning 
algorithms  presented  in  Sections  5.2.5  and  5.2.6.  Besides  the  fact  that  each  agent  was 
programmed  to  expect  different  sensor  patterns,  the  main  logic  of  the  IL  algorithm  for 
both  agents  was  the  same.  Although  Talrik  II™  and  Mantaray  have  very  different  sensor 
layout,  the  same  IL  algorithm  was  able  to  autonomously  configure  the  motor  directions 
and  configure  the  sensors  to  their  proper  port. 

As  expected,  the  performance  of  motor  configuration  algorithm  was  very  robust. 
Even  after  attempts  were  made  to  hinder  the  algorithm  (only  in  the  motor  configuration 
algorithm),  the  algorithm  stayed  true  to  form.  In  one  experiment,  obstacles  of  various 
sizes  and  shapes  were  introduced  into  the  environment.  It  was  discovered  that  as  long  as 
the  obstacles  were  placed  near  the  edge  of  the  environment,  the  motor  configuration 
algorithm  performed  very  well. 

The  sensor  configuration  algorithm  worked  well,  but  it  was  not  as  robust  as  the 
motor  configuration  module.  The  main  culprit  was  the  presence  of  noise  and  interference 
in  the  low-cost  infrared  proximity  sensors.  Due  to  the  presence  of  noise  in  the  sensors, 
Talrik  II™’ s performance  was  less  robust  than  that  of  Mantaray.  If  one  considers  the 
difference  in  density  of  sensors  between  Mantaray  and  Talrik  II™,  one  can  understand 
why  Talrik  II™  would  have  more  trouble  in  properly  configuring  its  sensors.  As  sensors 
are  packed  into  smaller  and  smaller  field  of  coverage,  they  start  to  overlap.  For  example. 
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Mantaray  would  detect  an  obstacle  to  the  right  with  only  its  right  sensor,  while  Talrik  II™ 
would  detect  it  with  two  or  three  sensors.  Consequently,  the  higher  density  of  sensors 
results  in  smaller  separations  of  the  GAP  time.  Although  it  was  seldom,  introduction  of 
noise  caused  improper  configuration  of  the  sensors  for  Talrik  II™,  while  Mantaray  did 
not  suffer  from  this  problem.  However,  this  problem  was  corrected  by  using  multiple 
GAP  time  to  eliminate  the  incorrect  GAP  time  pattern.  For  example,  three  separate  GAP 
time  patterns  were  created.  First  GAP  time  pattern  was  created  from  the  maximum 
reading.  Second  was  created  from  when  the  sensors  first  detect  the  wall.  Finally,  third 
was  created  from  when  the  sensor  stopped  detecting  the  wall  as  the  wall  got  out  of  range 
of  the  sensor  in  question.  The  success  of  the  two  IL  algorithms  was  verified  by  observing 
the  good  performance  of  the  obstacle  avoidance  program  on  Talrik  II™  and  Mantaray. 

5.4  Conclusion 

By  taking  advantage  of  simple  innate  knowledge  about  the  environment  and  their 
physical  embodiment,  Mantaray  and  Talrik  II™  were  able  to  properly  configure  the 
actuators  and  the  sensors.  The  use  of  innate  knowledge  parallels  the  animal  world,  in  that 
some  knowledge  is  important  enough  to  be  hard  coded  into  their  genes. 

Two  Innate  Learning  algorithms,  motor  configuration  and  sensor  configuration, 
were  implemented  and  tested  on  two  physically  different  autonomous  mobile  agents, 
Talrik  II™  and  Mantaray.  The  IL  algorithms,  inspired  by  natures  own  solutions, 
demonstrated  that  an  agent  could  gain  “awareness”  of  its  actuators  and  its  sensor. 
Although  their  sensor  layouts  were  significantly  different,  the  two  IL  algorithms  were 
able  to  properly  configure  their  actuators  and  sensors.  Furthermore,  this  section  showed 
how  innate  knowledge  (simple  knowledge  about  the  environment  and  the  agent’s  own 


35 


physical  embodiment)  can  greatly  improve  and  simplify  the  agent’s  ability  to  learn.  This 
parallels  the  animal  world,  in  that  some  inherent  knowledge  is  important  enough  to  be 
hard  coded  into  their  genes. 


CHAPTER  6 

ENVIRONMENTAL  REINFORCEMENT  LEARNING  (ERL) 

This  section  presents  Environmental  Reinforcement  Learning  (ERL):  a real-time 
learning  architecture  for  refining  primitive  behaviors  through  repetitive  execution  of 
those  behaviors  in  highly  structured  environments.  Environment  plays  an  essential  role 
in  ERL  not  only  by  providing  a suitable  reinforcement  to  the  agent,  but  also  by  forcing 
the  agent  to  execute  the  behavior  repetitively.  The  strength  of  ERL  lies  in  the  fact  that 
complex  factors  governing  primitive  behaviors  can  be  reduced  or  simply  ignored  by 
treating  the  agent’s  behavioral  interactions  with  the  environment  as  a “black  box.”  In 
other  words,  analysis  of  different  sources  of  variance  in  a behavior  is  not  necessary  to 
refine  that  behavior.  The  specially  constrained  environment  “punishes”  or  “rewards”  the 
agent  for  incorrect  or  correct  execution  of  a behavior.  Any  reward  or  punishment  for  a 
behavior  is  relayed  to  the  agent  as  Environmental  Reinforcement  (ER),  thereby  causing 
the  agent  to  adjust  its  behavioral  parameters.  An  autonomous  mobile  agent,  Mantaray, 
experimentally  demonstrates  ERL  architecture  on  two  primitive  behaviors,  (1)  traversing 
in  a straight  line,  and  (2)  turning  180  degrees. 

6.1  Introduction 

A current  trend  in  autonomous  agent  learning  deals  with  optimally  sequencing 
certain  combinations  of  primitive  behaviors,  which  require  low  level  motor  skills  to  reach 
a goal.  There  have  been  many  advances  in  this  area  [35],  [41],  [43],  [45].  However,  the 
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learning  process  of  the  primitive  behaviors  themselves  should  not  be  ignored  or 
trivialized.  Even  if  a learning  algorithm  determines  the  proper  behavior  sequence  to 
reach  a particular  goal,  the  crude  accuracy  of  the  primitive  behaviors  can  substantially 
degrade  its  overall  performance.  Although  the  accuracy  of  certain  behaviors  can  be 
refined  using  more  precise  sensors  and  actuators,  one  pays  the  price  of  significantly 
higher  financial  cost  for  the  agent. 

In  practice,  characteristics  of  each  agent  are  different;  and  consequently,  the 
agent’s  physical  performance  varies  greatly  one  to  another.  Even  if  equivalent  parts  are 
used  to  assemble  two  seemingly  identical  agents,  the  actual  performance  of  these  two 
agents  can  vary  greatly.  In  fact,  possible  sources  of  errors  are  almost  limitless.  For 


example,  variance  can  be  introduced  in  the  radius  of  wheels,  mounting  of  the  motors, 


Figure  6. 1 Relative  error  vs.  the  number  of  revolutions,  where  R0ff  equals  the 
difference  in  radii  of  the  two  wheels. 
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As  shown  in  Figure  6.1  [53],  minute  difference  in  wheel  radius  can  result  in 
significant  accumulation  of  position  error  of  an  agent.  One  possible  solution  to  this 
problem  is  to  account  for  all  the  possible  sources  of  variance.  This  course  of  action 
would  be  tedious,  time  consuming,  and  practically  impossible.  Furthermore,  this  process 
must  be  repeated  for  each  new  agent. 

This  section  describes  a real-time  architecture  by  which  an  agent  can  learn  to 
correct  the  problems  encountered  in  the  accuracy  or  consistency  of  primitive  behaviors. 
This  architecture,  called  Environmental  Reinforcement  Learning  (ERL),  is  a learning 
process  where  the  agent  refines  the  primitive  behavior  through  multiple  iterations  of  the 
behaviors  to  be  refined.  The  agent  executes  the  behaviors  in  a highly  structured 
environment  from  which  the  agent  learns  through  Environmental  Reinforcement  (ER). 
Section  6.2  describes  and  provides  an  overview  of  ERL  architecture,  assumptions  taken, 
environmental  considerations,  mutual  refinement,  and  strategy  for  faster  convergence. 
Section  6.3  presents  experimental  results  and  analysis  of  physically  implemented  ERL 
architecture.  The  section  also  introduces  the  autonomous  mobile  agent,  Mantaray  [13], 
and  two  specific  examples  of  the  ERL  process  by  which  the  primitive  behaviors  are 
refined  (traverse  a straight  line  and  turn  180  degrees).  Finally,  Section  6.4  offers 
concluding  remarks. 

6.2  Environmental  Reinforcement  Learning 

As  shown  in  Figure  6.2,  Environmental  Reinforcement  Learning  (ERL)  defines  a 
learning  architecture  by  which  primitive  behaviors,  like  low  level  motor  skills  (turning  a 
certain  number  of  degrees,  traversing  in  a straight  line,  wall-following,  etc.),  are  refined 
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Figure  6.2  The  Environmental  Reinforcement  Learning  (ERL)  Architecture 

through  repetitive  execution  of  the  same  primitive  behaviors  in  a highly  structured 
environment.  The  environment  forces  the  agent  to  perform  a set  of  behaviors,  where  the 
execution  of  the  behaviors  results  in  Environmental  Reinforcement  (ER).  Due  to  the 
iterative  nature  of  ERL,  the  environment  should  be  designed  with  repetition  in  mind. 
This  requirement  can  usually  be  satisfied  by  having  an  enclosed  environment  with  a 
restrictive  path  for  the  agent.  As  long  as  the  agent  can  return  to  its  initial  starting  state,  an 
iterative  process  can  be  guaranteed.  Furthermore,  environmental  constraints  should 
induce  “punishment”  or  “reward”  in  the  agent  for  incorrect  or  correct  execution  of  a 
behavior.  Any  reward  or  punishment  conveyed  to  the  agent  in  this  manner  is  ER.  ER 
stems  from  sensory  input  triggered  by  environmental  constraints,  such  as  a proximity 
sensor  indicating  an  approaching  wall.  The  punishment  induced  by  the  environment 
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causes  the  agent  to  adjust  learning  behavioral  parameters  for  the  behavior  being 
punished.  Consequently,  the  agent  learns  to  perform  the  behavior  “better”  on  the  next 
trial. 

6.2.1  Assumptions 

Although  ERL  architecture  can  be  adapted  for  higher  level  learning  with  or 
without  increased  complexity,  it  is  best  suited  for  primitive  behaviors.  ERL  favors 
primitive  behaviors  due  to  the  following  assumptions.  First,  the  variance  of  a behavior  is 
constant  (time-invariant)  for  a given  state  of  the  agent.  Second,  the  variance  of  a 
behavior  depends  only  on  the  practically  (easily)  observable  states  of  the  agent.  Finally, 
a simple  environment  can  be  configured  to  force  the  execution  of  the  desired  behavior 
and  ER  can  be  given  to  the  agent  as  feedback.  These  are  quite  restrictive  assumptions, 
especially  when  dealing  with  complex  behaviors.  However,  for  primitive  behavior  these 
assumptions  become  quite  reasonable.  In  fact,  ERL  can  be  applicable  even  for  primitive 
behaviors,  which  fail  the  first  or  second  assumptions,  by  treating  the  agent’s  behavioral 
interactions  with  the  environment  as  a “black  box.”  In  other  words,  analysis  of  different 
sources  of  variance  in  the  behavior  is  not  necessary  to  refine  a behavior  within  acceptable 
error.  Consequently,  one  may  only  be  interested  in  what  actuation  control  is  necessary 
for  proper  execution  of  a behavior  without  having  to  understand  the  sources  of  error  in 
that  behavior. 

6.2.2  Environmental  Reinforcement  (ER)  and  Environmental  Considerations 

Environment  plays  a critical  role,  since  the  agent  learns  through  its  interaction 
with  the  environment.  First,  the  environment  must  be  configured  in  such  a manner  that  it 
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forces  the  agent  to  perform  the  desired  behaviors.  Second,  for  each  forced  behavior,  the 
agent  must  receive  ER,  which  the  agent  interprets  as  reward  or  punishment.  ER  allows 
the  agent  to  evaluate  its  performance  for  that  behavior  and  to  adjust  the  learning 
behavioral  parameters  accordingly.  One  method  of  establishing  ER  is  to  specify 
restrictive  paths  for  the  behaviors  to  be  learned.  For  perfect  behavioral  parameters  and 
ideal  hardware,  the  agent  does  not  receive  any  negative  ER  from  the  environment. 
However,  due  to  non-ideal  variation  in  agent’s  behavior,  an  agent  receives  ER  for  any 
deviation  from  the  environmental  constraints.  Ultimately,  the  agent  must  return  to  the 
initial  position  for  repeated  runs  under  the  same  environmental  conditions.  This  ability  to 
find  the  initial  position  guarantees  continual  repetition  of  behaviors  for  learning  purposes. 

6.2.3  Mutual  Refinement  Process 

An  agent's  ability  to  refine  one  primitive  behavior  is  usually  dependent  on  the 
refinement  of  other  primitive  behaviors.  These  dependent  behaviors  require  mutual 
refinement  process  of  several  behaviors.  By  executing  the  interdependent  behaviors  in 
one  series  of  iteration,  mutual  refinement  of  several  behaviors  can  be  accomplished. 
However,  the  Mutual  Refinement  Process  assumes  that  the  environmental  constraints 
allow  for  the  required  combination  of  mutual  behaviors.  Consequently,  the  learning 
performance  can  be  greatly  enhanced  by  learning  the  interdependent  behaviors 
concurrently. 

6.2.4  Faster  Convergence  through  “Fine”  and  “Rough”  Adjustment  Trials 

By  starting  with  the  “rough”  adjustment  trials,  one  can  achieve  faster  convergence 
of  the  behavioral  parameters.  The  desired  characteristic  of  the  “rough”  adjustment  trials 
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is  to  minimize  the  time  necessary  to  execute  one  iteration,  correct  by  larger  adjustment 
values,  or  both.  Once  “rough”  adjustment  trials  converge  (either  a set  condition  or 
oscillatory  condition  occurs),  “fine”  adjustment  trials  begin.  “Fine”  adjustment’s  main 
goad  is  to  fine-tune  the  behavioral  parameter  as  accurately  as  possible. 

6.3  Experimental  Result 

The  Environmental  Reinforcement  Learning  (ERL)  architecture  was  tested  for  the 
following  two  primitive  behaviors:  traversing  in  a straight  line  and  turning  a specified 
amount  of  degrees.  Both  behaviors  were  implemented  concurrently  in  both  the  “rough” 
and  “fine”  adjustment  trials  on  Mantaray. 

6.3.1  Mantaray:  Autonomous  Mobile  Agent 

For  the  realization  of  the  ERL  architecture,  Mantaray  was  used  as  an  autonomous 
mobile  robot  platform.  For  a detailed  description  of  Mantaray,  refer  back  to  Section  4.2. 
Mantaray  is  an  autonomous  mobile  robot  developed  for  general  experiments  in  1995. 
However,  several  modifications  were  made  to  the  original  body  and  circuitry.  First,  the 
rolling  caster  was  replaced  by  a stationary,  sliding  caster.  The  rolling  caster  was  replaced 
due  to  the  unpredictable  error  it  introduced.  Second,  a voltage  divider  was  added  in 
parallel  to  the  battery  voltage  with  an  analog  tap;  thus,  allowing  the  agent  to  sense  the 
battery  voltage. 

6.3.2  Learning  to  Traverse  a Straight  Line 

Due  to  numerous  sources  of  errors  as  mentioned  in  the  introduction,  Mantaray 
had  a tendency  to  arc  left  with  the  same  speed  (Pulse  Width  Modulation)  applied  to  both 
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ER  Hit  Left 
ER  Safe  Center 
ER  Hit  Right 

(a)  (b) 

Figure  6.3  Environmental  setup  for  the  experiment. 

(a)  Environmental  Reinforcement  (ER)  for  straight  line  traversing. 

(b)  Actual  picture  of  the  environment. 

motors.  The  tendency  to  stray  from  the  straight  path  was  controlled  by  the  speed 
coefficient.  The  speed  coefficient  represents  a percentage  by  which  the  faster  motor  is 
slowed  to  match  the  speed  of  other  motor.  The  environmental  constraints  applied  to 
Mantaray  was  a straight  corridor,  shown  in  Figure  6.3,  with  width  of  18  inches  and  length 
of  30  feet  for  the  “fine”  adjustment  trials  and  length  of  10  feet  for  “rough”  adjustment 
trials.  The  two  ends  of  the  corridor  were  blocked  off.  The  Strabismic  sensors  were  used 
to  keep  Mantaray  in  the  center  of  the  corridor,  and  since  the  corridor  was  straight, 
Mantaray  was  forced  to  traverse  in  a straight  path.  ER  occurred  when  Mantaray  veered 
off  its  center  path  and  violated  the  ER  zone  to  the  left  or  to  the  right.  Figure  6.3  (a)  gives 
a pictorial  view  of  the  ER  zones.  As  Mantaray  traversed  the  full  length  of  the  corridor,  it 
counted  the  number  of  ER  from  the  left  and  number  of  ER  from  the  right.  If  Mantaray 
had  more  ER’s  on  one  side  than  the  other  with  a difference  of  two  or  greater,  the  speed 
coefficient  was  adjusted  to  equalize  the  speed  of  both  sides.  The  stop  condition  occurred 


when  the  front  sensor  detected  the  end  of  the  corridor. 
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6.3.3  Learning  to  Turn  180  Degrees 


(d)  Right  First  Hit  <e>  (f)  Left  First  Hit 


Figure  6.4  Environmental  Reinforcement  (ER)  for  Turn  180  degrees. 

The  same  straight  corridor  was  also  used  to  learn  the  turning  coefficient.  Turning 
coefficient  represents  the  absolute  time  to  turn  180  degrees.  When  Mantaray  reaches  the 
end  of  the  corridor  after  a straight  behavior,  it  ends  up  parallel  to  the  corridor  facing  the 
enclosed  end  of  the  corridor,  as  in  Figure  6.4(a).  This  forces  a 180-degree  turn  by 
Mantaray  in  order  to  face  the  open  end  of  the  corridor.  Initial  turning  reference  is 
obtained  by  measuring  the  actual  time  it  takes  Mantaray  to  turn  until  the  front  sensor 
detects  open  space.  Since  motor  speed  is  dependent  not  only  on  the  duty  cycle  of  the 
pulse  width  modulation  but  also  on  the  battery  voltage  applied  to  the  motors,  the  learned 
turning  coefficient  is  a function  of  the  battery  voltage  measured  during  the  180  degree 
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turn.  Once  Mantaray  obtains  the  turning  reference,  it  turns  for  an  amount  of  time 
specified  by  the  turning  coefficient,  as  shown  in  Figure  6.4(b),  and  stops  in  a position 
shown  in  Figure  6.4(c)  or  (e).  Finally,  Mantaray  goes  forward  until  an  increase  (decrease 
in  distance  to  the  wall)  in  one  of  the  Strabismic  sensor  is  detected,  as  shown  in  Figure 
6.4(d)  or  (f).  This  detection  then  becomes  the  ER  for  this  behavior.  Depending  on 
whether  ER  occurred  on  the  left  or  on  the  right,  the  turning  coefficient  is  appropriately 
decreased  or  increased. 

6.3.4  Speed  and  Turning  Offset  (Adjustment  Values) 

Initially,  speed  offsets  are  set  to  1%  and  turning  offsets  are  set  to  50  milliseconds. 
The  initial  values  of  the  offsets  are  not  tweaked  values.  Except  for  the  obvious  extremes 
like  zero,  the  offset  can  take  on  just  about  any  value.  If  the  offsets  start  as  small 
numbers,  it  would  take  the  agent  longer  to  reach  the  correct  value  but  would  settle 
quickly  with  minimal  oscillation.  On  the  other  hand,  if  the  offsets  start  as  large  numbers, 
it  would  take  the  agent  less  time  to  reach  the  correct  value  but  due  to  oscillation,  would 
settle  slowly.  The  values  of  each  offset  do  not  change  as  long  as  the  “trend”  does  not 
change  (i.e.,  consistently  increasing  or  decreasing).  However,  as  soon  as  the  “trend” 
changes,  the  offset  is  reduced  to  half  of  its  previous  value.  This  strategy  allows  for  fast 
convergence  in  the  beginning  when  the  coefficients  are  far  from  their  true  values.  On  the 
other  hand,  as  the  coefficients  near  their  true  values,  large  offsets  cause  over  adjustments 
which  in  turn  causes  “trend”  changes.  Consequently,  adjustments  become  finer  and  finer 
as  the  offsets  decrease.  To  keep  the  adjustments  meaningful,  lower  limits  of  0.01%  and  1 
millisecond  were  set  for  the  speed  and  turning  offsets,  respectively. 
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6.3.5  Mutual  Refinement 

As  the  speed  coefficient  approaches  its  correct  value,  the  initial  condition  for  the 
turning  behavior  becomes  more  stable.  Furthermore,  being  able  to  go  straight  ensures 
that  the  ER  for  the  turning  coefficient  does  not  occur  due  to  incorrect  arcing  of  Mantaray. 
Similarly,  as  the  turning  coefficient  approaches  its  optimal  value,  the  initial  condition  for 
the  turning  behavior  becomes  more  stable. 

6.3.6  Experimental  Results  and  Analysis 

The  state  of  the  agent  and  the  learning  coefficients  were  recorded  into  the  agent’s 
on  board  memory  after  completion  of  every  trial.  The  following  data  were  collected  at 
the  end  of  every  trial: 

1.  Current  system  time  (seconds). 

2.  Current  battery  voltage  (analog  sensor  reading  in  the  range  of  225  to  197,  which 
translate  into  10.71V  to  9.38V). 

3.  Current  Speed  coefficient  (percent)  used  to  decrease  the  faster  motor  to  equalize 
the  speed  of  both  motors. 

4.  Current  Turning  coefficient  (milliseconds)  used  to  turn  for  a set  amount  of  time 
for  180  degree  turn. 

5.  Update  Speed  coefficient  as  a function  of  battery  voltage. 

6.  Update  Turning  coefficient  as  a function  of  battery  voltage. 

Due  to  the  Mutual  Refinement  dependencies,  the  “rough”  trials  were  used  for  the 
first  99  trials.  Consequently,  the  coefficients  were  updated  three  times  as  fast.  However, 
“rough”  trials  using  a short  corridor  causes  problems  for  the  speed  coefficients,  because 
the  agent  does  not  have  enough  corridor  length  to  execute  the  straight  behavior, 
especially  as  the  turning  becomes  more  accurate.  Due  to  the  poor  result  of  speed 
coefficients  during  the  “rough”  trials,  the  speed  offsets  were  reset  to  their  initial  values  at 
the  beginning  of  the  “fine”  trials.  This  transition  also  points  out  the  Mutual  Refinement 
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Turning  Coefficient  vs.  Trials  (Battery  = 9.81  Volts)  Speed  Coefficient  vs.  Trials  (Battery  = 9.81  Volts) 


Turning  Coefficients  vs.  Battery  Voltage 


Speed  Coefficients  vs.  Battery  Voltage 


Battery  Voltage  (Volts)  Battery  Voltage  (Volts) 

(c)  (d) 


Figure  6.5  Summary  of  experimental  data  for  Turning  and  Speed  coefficients.  Trials 
one  to  99  represent  “rough”  trials.  The  rest  represent  “fine”  trials 
(a)  and  (b):  Turning  and  Speed  coefficients  vs.  Trials  are  plotted  for  a 
single  battery  voltage  level  of  9.81  Volts 

(c)  and  (d)  represent  the  accumulated  Turning  and  Speed  coefficients  for 
each  battery  level. 


process.  As  the  speed  coefficient  over  adjusts,  the  turning  coefficient  is  improperly 
affected,  as  seen  in  the  hump  that  occurred  in  the  turning  coefficient  after  the  99th  trial. 
However,  as  the  speed  coefficient  settles  down,  so  does  the  turning  coefficient. 

In  Figure  6.5  (c)  and  (d),  inconsistent  fluctuations  in  the  coefficients  above  battery 
voltage  of  10V  is  due  to  the  small  number  of  trial  runs  for  these  battery  voltages.  Eight 
NiCad  batteries  should  produce  9.6V  when  fully  charged,  but  the  batteries  actually 
charge  up  to  10.7V.  However,  the  batteries  can  not  hold  this  charge  for  very  long;  thus. 
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resulting  in  small  number  of  trial  runs  at  the  voltages  higher  than  10V.  Keeping  this  in 
mind,  we  concentrate  in  the  interval  between  9.38V  and  10.0V. 

The  performance  of  the  Speed  Coefficient  was  disappointing.  It  raised  more 
questions  than  answers.  Even  when  the  motor’s  pulse  width  modulation  was  directly 
controlled  through  interrupts  without  any  other  overhead  through  the  use  of  assembly 
code  and  the  speed  coefficient  was  manually  tweaked,  the  Mantaray’s  straight  path 
improved,  but  was  still  unpredictable.  This  meant  that  the  performance  of  the  motors 
varied  as  a function  of  some  factor  that  was  poorly  accounted  for.  This  variance  was 
great  enough  that  although  ERL  improved  the  behavior,  consistency  was  not  achieved. 

On  the  other  hand,  good  results  were  obtained  for  the  Turning  Coefficients.  The 
180-degree  turns  were  executed  with  precision  where  the  error  was  less  than  five  degree. 
Linear  regression  analysis  on  the  data  points  less  than  or  equal  to  10  Volts  resulted  in 
standard  error  of  19.56  milliseconds.  This  translates  into  standard  error  of  1.2  degrees  for 
a 180-degree  turn. 

6.4  Conclusion 

In  this  section,  two  primitive  behaviors  were  developed  using  the  ERL 
architecture.  This  architecture  proposes  a novel  idea,  where,  the  environment  forces  an 
agent  to  alter  its  behavior  in  a desired  manner.  Through  ER,  the  agent  learns  to  modify 
and  refine  its  behavior.  Furthermore,  complexities  were  reduced  by  treating  the  agent’s 
behavioral  interactions  with  the  environment  as  a “black  box.” 

Refinement  of  two  primitive  behaviors,  traversing  in  a straight  line  and  turning 
180  degrees,  was  attempted.  The  first  behavior  showed  significant  improvements  when 
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compared  to  the  original  performance  but  lacked  consistency.  The  180  degrees  turning 
behavior,  however,  showed  the  flexibility  and  simplicity  of  the  ERL  architecture. 

Possible  future  work  in  the  area  of  ERL  includes  turning  in  subdivisions  of  180 
degrees  (i.e.,  90,  45,  30  degrees).  In  addition,  by  adjusting  the  speed  coefficient  to  keep 
the  motor  at  constant  absolute  speed,  the  motor  speed  can  be  independent  of  changes  in 
the  battery  voltage. 

As  primitive  behaviors  are  refined  through  ERL , the  problems  of  object 
identification  and  map  building  can  be  simplified.  Wonder  what  an  agent  can  do  if  it 
could  measure  angles  and  distances  with  negligible  or  small  error?  These  two  problems 
are  definite  candidate  applications  of  the  behaviors  refined  through  ERL. 


CHAPTER  7 

AUTONOMOUS  EVOLUTION  OF  SENSORY  AND  ACTUATOR  DRIVER  LAYERS 
THROUGH  ENVIRONMENTAL  CONSTRAINTS 


Although  future  applications  for  autonomous  mobile  robots  (AMR)  are  practically 
limitless,  researchers  must  first  address  some  of  the  hurdles  blocking  wide  acceptance  of 
AMR  as  a viable  solution.  This  research  addresses  some  of  these  issues  through  the 
realization  of  Innate  Learning  (IL)  presented  in  Chapter  5 [15],  Environmental 
Reinforcement  Learning  (ERL)  presented  in  Chapter  6 [14],  and  Autonomous  Evolution 
of  sensory  and  actuator  Driver  layers  through  Environmental  Constraints  (AEDEC). 
Innate  Learning  (IL)  is  a learning  mechanism  that  takes  advantage  of  innate  knowledge  to 
improve  and  enhance  learning.  Using  innate  knowledge  of  it  embodiment  and  its 
environment,  IL  provides  a simple  mechanism  to  autonomously  detect  and  correct 
discrete  production  errors  (i.e.,  errors  in  wiring  of  sensors  and  actuators).  Environmental 
Reinforcement  Learning  (ERL)  is  a real-time  learning  architecture  for  refining  primitive 
behaviors  through  interaction  within  highly  structured  environments.  ERL  architecture 
allows  self-calibration  of  sensors,  actuators,  and  primitive  behaviors  by  using  a structured 
environment  (i.e.,  an  obstacle  course)  to  provide  real-time  feedback  on  a robot’s 
performance.  Through  the  refinement  process,  lower  cost  parts  can  be  used  and  damaged 
parts  can  be  replaced  without  affecting  the  rest  of  the  system.  Finally,  AEDEC  learning 
architecture  is  the  culmination  of  the  previous  research,  namely  Innate  Learning  (IL)  and 
Environmental  Reinforcement  Learning  (ERL).  By  incorporating  IL  and  ERL,  AEDEC 
asserts  that  sensory  and  actuator  driver  layers  can  be  autonomously  evolved  from  a 
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simple  set  of  innate  knowledge  guided  by  static  constraints  from  a highly  structured 
environment.  Using  innate  knowledge  and  a highly  structured  environment,  AEDEC 
allows  a robot  to  autonomously  create  abstractions  (drivers)  of  sensory  information  and 
actuation  controls,  consequently,  reducing  the  workload  of  a human  programmer.  Since 
different  type  of  robots  (walking,  two  wheels,  caterpillar  treads,  etc.)  can  be  trained  in  the 
same  environment,  AEDEC  permits  code  (high-level  behavior)  portability  between 
different  types  of  robots. 

7.1  Introduction 

The  personal  computer  (PC)  industry  has  gone  through  tremendous  changes  in  its 
relatively  short  existence.  It  started  as  a novelty  to  extreme  hobbyists  and  scientists  and 
has  evolved  to  become  an  essential  tool  for  most  today.  Many  factors  contributed  to  the 
current  success  of  the  PC  industry,  but  one  of  these  factors  has  had  a profound  effect  on 
the  mass  populace’s  view  of  the  PC.  With  the  advent  of  operating  systems,  users  no 
longer  needed  to  worry  about  specific  features  or  properties  of  the  individual  pieces  of 
hardware  that  makes  up  the  modern  day  PC.  Instead,  by  the  use  of  drivers,  operating 
systems  made  it  possible  for  users  to  treat  all  printers,  monitors,  modems,  etc.  the  same. 
The  drivers  allowed  vastly  differing  hardware  to  look  and  act  the  same  to  the  user. 
Although  this  is  a simplified  view,  the  point  has  been  made  that  by  abstracting  the 
detailed  low  level  control  of  each  hardware  device  to  a higher  level  command  (i.e.,  print), 
usability  of  the  device  is  greatly  enhanced. 

For  a technology  or  a product  to  be  widely  accepted  and  used,  it  must  first 
become  affordable  to  the  masses  and  be  robust  enough  for  novices  to  use.  Autonomous 
mobile  robots  have  yet  to  meet  these  criteria.  In  recent  years,  autonomous  mobile  robots 
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have  been  introduced  into  the  children’s  toy  market.  Although  the  robotic  toys  are  very 
limited  in  its  capability,  they  were  relatively  cheap  and  were  able  to  withstand  the  rough 
treatment  of  their  owners.  The  success  of  autonomous  mobile  robots  not  only  depends  on 
innovative  advances  in  technology,  but  it  also  depends  on  its  viability  as  a consumer 
product,  namely  affordability  and  reliability. 

Assembly  lines  tend  to  make  manufacturing  of  a product  less  costly.  Many 
industries  have  benefited  by  introducing  assembly  line  to  their  manufacturing  process. 
Consequently,  the  same  type  of  assembly  line  mentality  must  be  introduced  to 
autonomous  mobile  robots.  Manufacturing  of  the  mechanical  body  and  the  electronics 
that  make  up  the  physical  robot  can  easily  be  ported  over  to  assembly  lines  that  exist 
today  and  are  widely  being  used  in  other  industries.  However,  programming  an 
autonomous  mobile  robot  has  yet  to  take  advantage  of  the  benefits  offered  by  assembly 
lines. 

7.2  AEDEC 

By  incorporating  IL  and  ERL,  AEDEC  asserts  that  sensory  and  actuator  driver 
layers  can  be  autonomously  evolved  from  a simple  set  of  innate  knowledge  guided  by 
static  constraints  dictated  by  a highly  structured  environment.  Normally,  sensory  and 
actuator  driver  layers  are  hand  coded  by  a programmer  and  meticulously  tweaked  for 
optimal  performance.  Tweaking  of  these  parameters  is  time  consuming  and  tedious. 
Furthermore,  hand  coded  drivers  are  prone  to  inconsistencies  between  different 
programmers  and  between  different  robot  platforms  [14].  For  example,  programmer  A 
might  consider  1 foot  as  close  and  accordingly  set  the  parameter  thresholds;  while, 
programmer  B might  consider  6 inches  as  close  and  set  the  parameters  differently.  Even 
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if  the  physical  platform  of  the  robot  was  the  same  (which  is  practically  impossible),  two 
robots  (one  programmed  by  A and  other  by  B)  will  behave  significantly  different  from 
each  other  for  a given  behavior.  Another  example  is  with  two  physically  different  robots. 
The  robots  might  have  different  types  of  sensors,  different  sensor  locations,  different 
body  shapes,  different  means  of  movement,  etc.  These  physical  differences  usually  lead 
to  drastically  different  drivers  being  written  for  each  robot. 

The  differences  in  the  sensory  and  actuator  drivers  are  irrelevant  in  the  context  of 
a single  robot.  The  problem  becomes  evident  when  dealing  with  multiple  robots. 
Inconsistencies  in  the  design  of  the  drivers  will  lead  to  inconsistent  results  from  multiple 
robots.  Even  if  the  robots  are  constructed  with  the  same  parts,  they  will  have  differences 
in  their  sensor  and  actuator  performances.  Although  one  can  get  around  this  problem  by 
using  high  precision  sensors  and  actuators,  this  strategy  will  greatly  increase  the  cost  of 
the  robots.  Another  solution  would  be  to  have  the  programmer  code  drivers  separately 
for  each  robot.  Again,  this  would  be  very  costly. 

Instead  of  a human  programmer  coding  the  driver  for  an  autonomous  mobile 
robot,  Autonomous  Evolution  of  sensory  and  actuator  Driver  layers  through 
Environmental  Constraints  (AEDEC)  states  that  drivers  should  be  evolved  from  a simple 
set  of  innate  knowledge  guided  by  static  constraints  dictated  by  the  highly  structured 
environment.  By  running  a robot  through  a highly  structured  obstacle  course,  robot’s 
interaction  with  the  environment  in  conjunction  with  preprogrammed  innate  knowledge 
would  evolve  the  drivers  with  the  constraints  dictated  by  the  environment. 
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7.2.1  Environment 

The  environment  plays  an  essential  role  in  AEDEC  as  it  did  in  ERL.  Given 
simple  innate  knowledge,  a robot  will  interact  with  the  highly  structured  environment  to 
evolve  and  refine  its  set  of  sensory  and  actuator  driver  layers.  Specifications  of  the 
drivers  are  dictated  by  the  environment  and  not  by  a programmer.  This  relieves  the 
burden  of  parameter  tweaking  from  a programmer  to  the  environment.  Consistency  of 
the  drivers  would  be  guaranteed  as  long  as  the  environment  does  not  change.  Although 
creating  an  environment  for  the  first  time  might  be  much  more  time  consuming  than  hand 
coding  a driver  for  a small  number  of  robots,  it  will  save  time  when  dealing  with  large 
number  of  robots.  Since  the  robots  will  evolve  their  own  drivers  by  completing  the 
obstacle  course  in  an  “assembly  line”  manner,  a programmer  will  be  free  to  do  other 
tasks.  Furthermore,  it  will  become  evident  in  the  following  sections  that  the  environment 
is  mostly  made  up  of  simple  geometric  shape  and  therefore,  can  easily  created  or 
duplicated. 

The  environment’s  primary  goal  is  to  present  scenarios  that  promote  evolution  or 
refinement  of  a robot’s  sensory  and  actuator  driver  layers.  This  can  be  accomplished  by 
modeling  the  environment  in  a form  of  an  obstacle  course.  An  obstacle  course  normally 
consists  of  a path  that  has  various  obstructions  along  the  path.  Obstructions  are  placed 
throughout  the  course  and  each  obstruction  is  designed  to  test  certain  abilities  of  the 
participant.  AEDEC  takes  a similar  approach.  AEDEC  forces  a robot  to  follow  a 
predetermined  path,  normally  by  outlining  an  obstacle  course  with  walls.  The  walls 
outlining  the  course  prevent  a robot  from  accidentally  wandering  away  from  the 
predetermined  path. 
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As  a robot  traverses  an  obstacle  course,  it  encounters  obstacles  in  its  path.  A 
robot  has  innate  (preprogrammed)  knowledge  about  each  of  the  obstacles  that  it 
encounters.  For  example,  the  first  obstacle  might  require  a Turn  180  (turn  180  degrees) 
maneuver  and  the  second  obstacle  might  require  a ForwardStraight  (go  forward  in  a 
straight  line)  maneuver.  If  a robot  performs  the  required  maneuver  properly,  it  will 
successfully  pass  through  an  obstacle.  If  not,  a robot  will  get  immediate  feedback 
(environmental  reinforcement)  from  the  environment,  and  accordingly  will  adjust  its 
Turnl80  parameters.  As  a robot  repeats  the  obstacle  course,  its  performance  on  each 
obstacle  improves  until  it  has  evolved  to  form  the  final  driver  for  a robot. 

7.2.2  Innate  Knowledge 

Innate  knowledge  plays  an  important  role  on  whether  some  animals  survive  or 
dies.  Certain  knowledge  or  behavior  is  so  critical  to  their  survival;  it  is  encoded  within 
their  genes,  almost  as  a reflex  behavior.  For  a detailed  discussion  on  innate  knowledge, 
refer  to  Section  5.2.3. 

What  is  innate  knowledge  and  what  is  not  innate  knowledge  in  the  context  of 
autonomous  mobile  robots  is  a complex  question  with  complex  answers.  This  is 
especially  true  when  using  the  definition  given  by  experts  in  natural  sciences.  Instead, 
AEDEC  uses  the  term  innate  knowledge  to  represent  simple  knowledge  that  is  hard 
coded  onto  a robot.  Although  AEDEC  follows  definitions  of  innate  knowledge  used  by 
scientists  in  natural  sciences,  AEDEC  does  not  make  any  formal  arguments  concerning 
each  knowledge  or  behavior  that  is  used  as  innate  knowledge. 

Some  examples  of  innate  knowledge  in  the  context  of  AEDEC  are  a robot’s 
knowledge  about  its  own  embodiment.  A robot  is  preprogrammed  know  how  to  access  it 
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actuators  and  sensors  at  the  hardware  level.  A robot  will  be  coded  with  the  ability  to  send 
out  control  signals  (direction,  PWM,  etc.)  to  the  appropriate  actuators  and  to  read  from 
sensors.  In  other  words,  a robot  will  be  working  with  abstracted  models  of  actuators  and 
sensors,  which  means  that  the  actuators  and  sensors  are  controlled  through  predetermined 
set  of  variables.  For  example,  a robot  knows  that  an  actuator  can  be  controlled  through 
its  motor  number,  direction,  and  speed.  However,  this  does  not  mean  that  a robot  would 
have  explicit  knowledge  that  correlates  certain  parameter  setting  with  certain  actions. 
Instead,  it  knows  that  certain  set  of  motors  control  its  movement.  It  still  needs  to  learn 
the  appropriate  parameters  to  cause  a forward  movement  at  6 inches  per  second.  This  is 
analogous  to  a baby  learning  how  to  throw  a ball. 

At  first,  a baby  has  barely  enough  coordination  to  pick  up  a ball,  let  alone  throw  a 
ball.  Baby’s  first  few  attempts  at  throwing  a ball  probably  results  in  a ball  being  dropped 
rather  than  being  thrown.  Initial  failed  attempts  are  useful  in  that  it  shows  what  not  to  do 
(i.e.,  search  for  the  proper  mechanics).  Ultimately,  a baby  will  learn  the  proper 
mechanics  of  throwing  a ball  and  improves  with  repetition. 

Another  category  of  innate  knowledge  in  AEDEC  is  the  environment.  As  stated 
in  Section  7.2.1,  environment  plays  an  essential  role  in  AEDEC.  For  example,  every 
time  a robot  is  put  into  an  environment,  a robot  has  preprogrammed  knowledge  about  the 
environment.  The  preprogrammed  knowledge  (innate  knowledge)  allows  a robot  to 
quickly  extract  useful  information  and  properly  associate  the  information.  Having  innate 
knowledge  about  an  environment  might  seem  inappropriate,  but  one  can  observe  many 
instances  in  nature.  Baby  sea  turtles  are  borne  with  innate  knowledge  about  its 
environment.  As  soon  as  it  hatches  out  of  its  shell,  it  digs  its  way  out  and  heads  straight 
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towards  the  safety  of  the  sea.  In  this  simple  behavior  to  survive,  one  can  see  the  sea 
turtles’  innate  knowledge  about  the  environment.  It  knows  that  it  must  dig  its  way  up  and 
out  of  the  sand  and  it  knows  that  it  must  get  to  the  sea  even  though  it  has  never  seen  it 
before.  In  this  one  example  of  sea  turtles,  one  can  observe  evidence  of  innate  knowledge 
in  the  animal  world. 

7.2.3  Effect  of  Highly  Structured  Environment  and  Innate  Knowledge  on  the 
Reinforcement  Learning  Search  Space. 

In  reinforcement  learning,  a robot  must  explicitly  explore  its  environment,  which 
can  be  a very  large  search  space.  Due  to  the  large  search  space  in  most  meaningful 
applications,  reinforcement  learning  takes  too  long  to  learn  (converge  to)  a goal.  AEDEC 
addresses  this  problem  by  incorporating  a highly  structured  environment  (from  ERL)  and 
innate  knowledge  (from  IL)  to  reduce  the  search  space  to  a more  manageable  size. 
Consequently,  AEDEC  is  a novel  hybrid  of  reinforcement  learning,  which  uses  the 
principles  of  reinforcement  learning  molded  by  highly  structured  environment  and  innate 
knowledge  to  temper  reinforcement  learning’s  unwieldy  search  space. 

7.2.4  AEDEC  Architecture 

AEDEC  architecture  is  actually  made  up  of  two  slightly  different  learning 
paradigms,  one  for  evolution  of  sensory  driver  layer  and  another  for  actuator  driver  layer. 
It  is  analogous  to  learning  how  to  recognize  different  types  of  fruits  and  learning  how  to 
play  basketball.  In  learning  how  to  recognize  different  types  of  fruits,  no  physical  action 
is  required;  instead,  the  subject  can  sit  in  a chair  and  have  different  fruits  shown  to  him. 
The  subject  needs  to  recognize  and  memorize  distinctive  features  in  each  fruit  that 
separates  it  from  all  the  other  fruits.  Whereas,  in  learning  how  to  play  basketball,  the 
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subject  is  trying  to  learn  and  refine  physical  skills  (shooting  free  throws,  jump  shot, 
dribbling,  etc.).  For  example,  the  subject  learns  to  shoot  free  throws  by  actually  shooting 
free  throws  until  he  learns  the  proper  mechanics  that  results  in  a made  free  throw. 
Obviously,  there  are  differences  between  recognizing  a particular  fruit  and  shooting  a 
basketball.  The  first  requires  extraction  of  useful  information  from  the  subject’s  senses 
and  the  second  requires  refinement  of  an  action  through  execution  of  the  action  in  a 
highly  structure  environment  (i.e.,  regulation  size  and  weight  of  basketball,  rim  size,  rim 
height,  etc.).  Therefore,  AEDEC  takes  on  slightly  different  forms  for  sensory  driver  layer 
and  actuator  driver  layer.  Although  the  learning  architectures  are  slightly  different,  they 
are  both  part  of  AEDEC,  since  both  share  inherent  properties  of  the  AEDEC  architecture. 
Both  architectures  use  innate  knowledge  and  a highly  structured  environment  to 
autonomously  evolve  a driver  layer,  which  in  essence  is  the  fundamental  principal  behind 
AEDEC. 

AEDEC  architecture  for  sensory  driver  layer,  shown  in  Figure  7.1,  describes  the 
relationship  between  a robot,  innate  knowledge,  and  structured  environment.  A robot 
would  first  make  a sensory  sweep  of  the  environment  and  note  the  changes  of  the  sensor 
readings  from  the  ambient  sensor  readings  (taken  in  an  empty  environment).  Then  it 
correlates  the  information  collected  about  the  environment  with  the  appropriate  innate 
knowledge  to  create  a sensor  template.  This  process  would  continue  until  all  the  sensor 
templates  are  created  and  verified. 

AEDEC  architecture  for  actuator  driver  layer,  shown  in  Figure  7.2,  describes  how 
high  level  actuation  commands  can  be  learned  and  refined.  Each  structured  environment 
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Figure  7.1  AEDEC  architecture  for  sensory  driver  layer. 
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Figure  7.2  AEDEC  architecture  for  actuator  driver  layer.  (IK  = Innate  Knowledge). 
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is  designed  to  give  a reward  or  punishment  for  executing  a particular  actuation.  For 
example,  a robot  performs  a Tumi 80  maneuver  in  an  environment  that  forces  that 
behavior.  Obviously,  a robot  must  learn  the  Turn  180  maneuver  that  follows  the  contour 
of  the  environment.  If  the  maneuver  is  executed  improperly,  a robot  would  end  up  hitting 
a wall  or  end  up  closer  to  one  wall  than  the  other  wall.  Depending  on  the  result,  a robot 
would  update  the  parameters  used  for  Turn  180  until  the  maneuver  is  executed  properly. 
Structure  environment  and  innate  knowledge  physically  shape  the  actuation  command 
and  provide  immediate  evaluation  of  the  actuation  command.  The  learning  process 
continues  until  all  the  actuation  parameters  are  finalized. 

7.3  Proposed  Experiment 

AEDEC  architecture  will  be  realized  on  two  different  autonomous  mobile  robots. 
Within  the  same  environment,  each  robot  will  go  through  the  process  of  learning  and 
creating  a sensory  and  actuator  driver  layers.  Then  a series  of  experiment  will  be 
performed  to  verify  the  performance  and  feasibility  of  the  AEDEC  architecture. 

In  the  realization  of  AEDEC,  two  different  autonomous  mobile  robots  (Talrik  II™ 
and  Mantaray)  will  be  used.  Mantaray  is  a two  wheel  based  robot  with  relatively  simple 
set  of  sensors;  while,  Talrik  II™  is  a two  wheel  based  robot  with  a relatively  rich  set  of 
sensors.  A more  detailed  discussion  of  each  robot  is  presented  in  Chapter  4. 


CHAPTER  8 

SENSORY  DRIVER  LAYER 


Sensors  are  a critical  component  of  an  autonomous  mobile  robot  (AMR). 
Sensors’  performance  and  the  way  sensors  are  utilized  can  greatly  affect  the  performance 
of  an  AMR.  In  many  cases,  sensors  are  painstakingly  characterized  and  studied  to  find 
the  proper  thresholds  and  use  of  the  sensors.  Although  it  is  time  consuming,  this  method 
has  worked  well  in  many  applications.  The  problem  arises  when  there  are  many  sensors. 
In  order  to  guarantee  the  same  performance  for  the  new  sensor,  every  new  sensor  had  to 
be  characterized,  since  different  sensors  can  have  different  characteristics.  The  problems 
are  further  compounded  in  the  cases  where  the  sensors  are  being  added  to  not  just  one 
AMR  but  multiple  AMRs  and,  even  worse,  multiple  AMRs  of  different  types.  When 
high  precision  sensors  are  used,  the  need  for  constant  characterization  of  sensors  can  be 
reduced.  The  problem  with  this  option  is  that  the  cost  of  the  AMR  increases. 

Even  if  high  precision  sensors  are  used,  sensor’s  performance  and  characteristics 
are  greatly  effected  by  how  the  sensor  is  physically  implemented  on  an  AMR.  For 
example,  an  infra-red  proximity  sensor’s  performance  is  greatly  effected  by  the 
orientation  of  the  sensor  on  the  AMR’s  platform.  The  variations  are  caused  by  different 
angles  respect  to  the  environment  and  interference  from  other  physical  parts  of  the  AMR 
causing  partial  reflections  and  partial  blockage. 

Now  imagine  trying  to  create  a sensory  driver  layer  for  a complex  sensor  suite  of 
an  AMR.  AMR  programmers  have  spent  many  tedious  hours  “tweaking”  the  sensor 
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parameters  of  a given  AMR  to  get  the  AMR  working  properly.  Although  tedious  and 
time  consuming,  it  has  worked  well  for  many  programmers.  The  problem  with  this 
method  is  that  the  programmer  was  constantly  “tweaking”  sensor  parameters.  Once 
normally  was  not  enough.  If  one  of  the  sensors  malfunctions  and  needs  to  be  replaced, 
programmer  had  to  recalibrate;  if  one  of  the  sensor’s  orientation  changed,  again 
recalibration  was  needed. 

Problems  do  not  end  with  tedious  characterizations.  When  a programmer  creates 
a driver  for  an  AMR,  the  driver  is  usually  robust  and  reliable.  However,  it  has  certain 
peculiarities  and  bias  of  the  original  programmer  built  into  the  drivers.  For  example, 
programmer  A’s  definition  of  close  might  mean  four  inches;  while,  programmer  B’s 
might  be  12  inches.  Both  versions  will  work  as  long  as  the  programmer  understands 
these  assumptions  taken  by  the  original  programmer  writing  the  drivers.  The  biggest 
problem  arises  when  the  sensors  are  changed,  damaged,  incorrectly  wired,  placed  in 
different  locations,  etc.  Take  the  instance  where  the  sensors  are  rearranged  to  meet  a new 
requirement.  This  might  result  in  sensor  1,  which  initially  pointed  forward,  pointing 
right.  With  other  sensors  also  changing  in  a similar  fashion,  it  would  require  total 
rewriting  of  the  drivers  for  the  robot,  since  sensor  1,  previously  used  to  detect  obstacles 
in  the  front,  can  no  longer  be  used  for  that  purpose.  Most  programmers  of  AMRs  have 
experienced  some  of  these  problems  in  one  form  or  another. 

A sensory  driver  layer  using  AEDEC  architecture  addresses  many  of  the  issues 
discussed  above.  The  programmer’s  work  of  creating  a driver  will  be  eliminated  by 
having  the  AMR  learn  the  driver  on  is  own.  If  sensors  change,  malfunctions,  or  even  if 
the  sensor  configuration  changes,  the  driver  can  be  updated  by  having  the  AMR  relearn 
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the  driver  over  again.  Furthermore,  different  AMRs  trained  using  AEDEC  will  have 
similar  interpretations  of  its  environment.  This  is  guaranteed  by  having  the  AMR  learn 
the  driver  in  the  same  highly  structured  training  environment.  Finally,  the  level  of 
abstraction  and  the  consistency  of  the  highly  structured  environment  will  allow  for  code 
portability. 

8.1  Innate  Knowledge  in  Sensory  Driver  Layer 

Certain  information  and  capabilities  are  hard  coded  onto  the  AMR.  The  AMR  has 
prior  knowledge  of  how  many  sensor  ports  it  has  and  how  it  can  get  raw  sensor  data  from 
those  ports.  However,  this  does  not  mean  that  the  AMR  is  preprogrammed  with  what 
type  of  sensors,  how  many  ports  actually  have  sensor  connected,  or  any  information 
about  thresholds  or  reading  ranges.  The  AMR  also  has  prior  information  about  the 
complete  list  of  the  sensor  scenarios.  For  example,  “object  to  the  front,”  “object  to  the 
far  front,”  “object  to  the  right,”  etc.  The  AMR  only  know  that  this  list  exist,  but  it  does 
not  have  any  information  about  the  properties  of  the  scenarios  in  the  list  or  what  they 
represent.  The  AMR  will  learn  the  meaning  of  these  scenarios  through  the  AEDEC 
architecture. 

8.2  Scenarios 

The  sensory  driver  layer  is  made  up  of  60  unique  scenarios.  Scenarios  are  highly 
structured  environmental  situations  presented  to  the  AMR  during  the  sensory  driver  layer 
learning  process.  Each  scenario  was  chosen  for  its  uniqueness  and  for  the  likelihood  of 
an  AMR  encountering  the  scenario  during  operation.  This  paper  does  not  make  any 
assertions  about  the  completeness  or  usability  of  the  given  set  of  scenarios.  The  set  of 
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scenarios  was  created  from  five  years  of  AMR  programming  experiences.  The  issue  of 
what  constitutes  the  best  set  of  scenarios  is  left  as  a future  research  topic. 


Table  8.1  The  complete  list  of  the  scenarios  and  descriptions 


Scenario  Name 

Description 

Scenario  Name 

Description 

NullTemp 

Empty  environment 

WBR30 

Wall  Back  Right  30  degrees 

FC 

object  Front  Close 

WBR30F 

Wall  Back  Right  30  degrees  Far 

F 

object  Front 

WLC 

Wall  Left  Close 

FRC 

object  Front  Right  Close 

WL 

Wall  Left 

FR 

object  Front  Right 

WLF 

Wall  Left  Far 

FLC 

object  Front  Left  Close 

WFL15C 

Wall  Front  Left  15  degrees  Close 

FL 

object  Front  Left 

WFL15 

Wall  Front  LefM  5 degrees 

LC 

object  Left  Close 

WFL15F 

Wall  Front  Left  15  degrees  Far 

L 

object  Left 

WFL30C 

Wall  Front  Left  30  degrees  Close 

RC 

object  Right  Close 

WFL30 

Wall  Front  Left  30  degrees 

R 

object  Right 

WFL30F 

Wall  Front  Left  30  degrees  Far 

BC 

object  Back  Close 

WBL15C 

Wall  Back  LefM  5 degrees  Close 

B 

object  Back 

WBL15 

Wall  Back  Left  1 5 degrees 

BRC 

object  Back  RightClose 

WBL15F 

Wall  Back  Left  1 5 degrees  Far 

BR 

object  Back  Right 

WBL30C 

Wall  Back  Left  30  degrees  Close 

BLC 

object  Back  Left  Close 

WBL30 

Wall  Back  Left  30  degrees 

BL 

object  Back  Left 

WBL30F 

Wall  Back  Left  30  degrees  Far 

WRC 

Wall  RightClose 

WFC 

Wall  Front  Close 

WR 

Wall  Right 

WF 

Wall  Front 

WRF 

Wall  Right  Far 

WFF 

Wall  Front  Far 

WFR15C 

Wall  Front  Right  1 5 degrees  Close 

CF 

Corner  Front 

WFR15 

Wall  Front  Right  15  degrees 

CFR 

Corner  Front  Right 

WFR15F 

Wall  Front  Right  15  degrees  Far 

CFL 

Corner  Front  Left 

WFR30C 

Wall  Front  Right  30  degrees  Close 

CR 

Corner  Right 

WFR30 

Wall  Front  Right  30  degrees 

CL 

Corner  Left 

WFR30F 

Wall  Front  Right  30  degrees  Far 

CBR 

Corner  Back  Right 

WBR15C 

Wall  Back  Right  15  degrees  Close 

CBL 

Corner  Back  Left 

WBR15 

Wall  Back  Right  15  degrees 

CB 

Corner  Back 

WBR15F 

Wall  Back  Right  15  degrees  Far 

DF 

Dead  end  Front 

WBR30C 

Wall  Back  Right  30  degrees  Close 

DB 

Dead  end  Back 

The  complete  list  of  the  scenarios  can  be  found  in  Table  8.1.  NullTemp  was 
created  by  having  an  empty  environment  and  is  useful  for  minimizing  detection  of 
“phantom”  objects  at  the  extreme  edges  of  the  detection  range  of  the  sensors.  The  next 
16  scenarios,  FC  to  BL,  were  created  by  placing  a 5 14  X 5 14  object  in  various  locations 
around  the  AMR.  The  directions  represented  by  the  16  scenarios  are  self  explanatory. 


65 


For  example,  F represent  front  of  the  AMR;  while  L represents  left  of  the  AMR.  The 
angle  between  F and  L would  be  90  degrees.  As  for  FL,  it  is  in  between  F and  L,  45 
degrees  from  each.  As  for  distance,  “far”  is  placed  18  inches  from  the  center  of  the 
AMR;  while,  normal  is  place  1 1 inches  from  the  center  of  the  AMR. 


Figure  8.1  Picture  of  WFL30F  scenario.  In  the  picture  Talrik  II™  is  facing  the  top  of 
the  picture. 

AMR  tend  to  encounter  “wall”  type  scenarios  more  frequently  and  it  is  more 
important  to  be  able  to  recognize  different  orientations  of  the  “wall”  than  a small 
obstacle.  Hence,  over  half,  33  to  be  exact,  are  “wall”  scenarios.  In  fact  the  remaining 
scenarios  can  also  be  classified  as  “walls,”  putting  majority  of  the  scenarios  in  the  “wall” 
category.  All  the  “wall”  scenario  names  start  with  W.  30  out  of  33  “wall”  scenarios  are 
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to  the  left  and  right  of  the  AMR  with  the  exception  of  three  in  the  front.  All  “wall” 
scenarios  were  created  using  a “wall”  24  inches  in  length.  WR  represents  a wall  that  is 
parallel  to  the  axis  line  running  from  the  front  to  the  back  of  the  AMR.  15  and  30 
degrees  represents  angle  of  the  “wall”  respect  to  WR  and  WL.  The  distance  represented 
by  “close,”  “normal,”  and  “far”  for  the  “walls”  are  nine,  11,  and  13  inches,  respectively, 
from  the  center  of  the  AMR.  An  example  of  a wall  scenario  is  shown  in  Figure  8.1. 


Figure  8.2  A picture  of  CBL  scenario.  In  the  picture  Talrik  II™  is  facing  the  top  of 
the  picture. 

“Comers”  are  implemented  using  two  12  inch  walls.  They  are  placed  at  90 
degrees  respect  to  each  other  to  create  a corner,  as  shown  in  Figure  8.2.  Each  of  the  walls 
making  the  comer  is  placed  1 1 inches  away  from  the  center  of  the  AMR.  The  direction 
represents  the  position  of  the  corner  of  the  walls  respect  to  the  AMR,  as  shown  in  Figure 
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8.2.  Final  category  of  scenarios  is  DF  and  DB.  In  the  case  of  DF,  three  12  inch  walls  are 
used  to  surround  the  front  of  the  AMR  to  create  a “dead  end.” 

There  are  enough  variations  in  the  given  set  of  scenarios,  to  represent  most  of  the 
real  world  environment  that  an  AMR  might  face.  Furthermore,  from  this  basic  set  of  60 
scenarios,  more  complex  scenarios  can  be  created  by  “merging”  two  or  more  scenarios 
together.  Due  to  time  constraints,  the  “merging”  property  will  be  left  as  a possible  future 
work. 

8.3  Learning  Algorithm  for  the  Sensory  Driver  Layer 

For  each  scenario,  the  appropriate  environment  is  created  according  to  the 
specifications  of  the  scenarios  listed  in  section  8.2.  The  AMR  is  programmed  to  ask  for  a 
scenario  and  then  the  scenario  is  presented  to  the  AMR.  The  AMR  takes  a complete 
reading  of  all  the  sensor  port  and  creates  a template  for  the  given  scenario.  This  process 
is  repeated  for  every  scenario  in  the  list.  For  this  experiment,  human  helper  laid  out  the 
scenarios  for  the  AMR.  But  for  future  implementations  an  automatic  environment 
controlled  by  the  AMR  would  lay  out  the  environment  for  each  scenario. 

In  order  to  create  the  templates,  six  different  readings  are  made  for  the  same 
scenario.  Then  for  each  sensor  port,  the  highest  and  the  lowest  values  are  dropped  and 
the  four  remaining  readings  are  averaged  to  create  each  scenario  template.  This  filtering 
process  was  implemented  to  minimize  the  environmental  noise  error,  sensor  fluctuations, 
etc. 

Once  the  scenario  templates  are  complete,  templates  could  be  used  to  find  the 
closest  match  for  any  new  scenario  that  the  AMR  might  encounter.  The  best  match  was 
found  using  a modified  Euclidean  distance.  For  example,  AMR  would  take  a sensor 
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sweep  of  all  it  sensors  and  then  find  the  modified  Euclidean  distance  of  the  new  scenario 
respect  to  all  of  the  scenarios  in  the  sensory  driver  layer.  The  scenario  with  the  smallest 
distance  is  then  selected  as  the  match.  The  modified  Euclidean  distance  is  given  in 
Equation  1,  where  N = number  of  sensors;  n = sensor  number;  X = database  of  template 
values;  Y = new  sensor  readings  to  be  matched. 

N- 1 

MED  = £(Xn  - Yn  f Equation  1 

n= 0 

8.4  Analysis  of  the  Sensory  Driver  Layer 

Once  the  scenario  templates  were  created,  series  of  experiments  were  performed 
to  test  the  performance  and  usability  of  the  sensory  driver  layer.  [39] 

8.4.1  Recognition  of  the  Original  Scenarios 

The  AMR  was  given  the  same  set  of  scenarios  to  see  if  it  could  properly  identify 
the  scenarios  that  it  learned.  In  order  to  analyze  the  data,  a table  of  modified  Euclidean 
distance  (MED)  was  made.  Complete  table  can  be  found  in  Appendix  A.  The  table 
given  in  Appendix  A is  60  by  60,  where  horizontal  headings  represent  the  scenarios  in  the 
database  and  the  vertical  headings  represent  scenarios  presented  to  the  AMR  for 
identification.  Each  cell  represents  the  MED  of  the  corresponding  vertical  and  horizontal 
headings.  For  readability,  the  cell  containing  the  smallest  MED  is  highlighted  with 
thicker  borders.  For  60  scenarios  tested,  the  AMR  correctly  identified  all  60  scenarios, 
represented  by  the  smallest  MED  occurring  on  the  diagonal  of  the  table. 
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IlNuHTem 

FC 

F 

FRC 

FR 

FLC 

FL 

LC 

L 

RC 

R 

BC 

B 

NullTem 

3 

1183 

206 

1010 

254 

1442 

224 

742 

164 

1296 

193 

737 

39 

FC 

1133 

9 

328 

1164 

962 

1206 

834 

1842 

1286 

2342 

1291 

1867 

1169 

F 

290 

372 

3 

871 

365 

1075 

269 

999 

443 

1583 

478 

1024 

326 

FRC 

999 

1261 

816 

4 

252 

2348 

1200 

1738 

1160 

1048 

759 

1733 

1035 

FR 

286 

1056 

325 

253 

3 

1707 

503 

1025 

447 

987 

276 

1020 

322 

FLC 

1331 

1031 

942 

2218 

1562 

18 

452 

1230 

1248 

2624 

1469 

2065 

1367 

FL 

307 

799 

246 

1278 

552 

476 

10 

716 

380 

1600 

475 

1041 

343 

LC 

713 

1859 

916 

1720 

964 

1058 

532 

12 

246 

2006 

869 

1447 

749 

L 

104 

1320 

331 

1079 

333 

1269 

227 

309 

11 

1365 

276 

838 

140 

RC 

1276 

2512 

1519 

1047 

971 

2783 

1523 

2015 

1437 

3 

492 

2010 

1312 

R 

203 

1371 

406 

746 

250 

1642 

424 

942 

364 

480 

5 

937 

239 

BC 

794 

2062 

1037 

1769 

1023 

2301 

1041 

1533 

955 

2055 

974 

4 

494 

B 

38 

1218 

241 

1045 

289 

1477 

259 

777 

199 

1331 

228 

472 

6 

Table  8.2  Partial  table  of  modified  Euclidean  distance  of  each  scenario  respect  to  the 
database  of  scenario  templates  from  Appendix  A 


For  easy  reference,  a small  part  of  the  table  given  in  Appendix  A is  shown  in 
Table  8.2.  In  Table  8.2  the  highlighted  cells  representing  matches  mostly  have  values  in 
the  single  digits,  while  other  cells  average  in  the  hundreds  and  sometimes  in  the 
thousands.  Larger  differences  represent  less  likelihood  for  misidentification.  For 
example,  reading  across  the  row  labeled  NullTemp  in  Table  8.2,  the  shaded  first  cell 
represents  the  match  between  NullTemp  and  NullTemp  with  a value  of  three.  The  next 
smallest  value  in  the  row  is  39,  representing  the  MED  between  NullTemp  and  B (object 
to  the  Back).  Although  39  is  an  order  of  magnitude  larger  than  the  matching  value  of 
three,  39  is  relatively  smaller  because  B is  a small  object  relatively  far  away  from  the 
AMR.  Consequently,  the  AMR  makes  a smaller  distinction  between  NullTemp  and  B 
than  between  NullTemp  and  BC  (object  to  the  Back  Close).  A detailed  description  of 


each  scenario  is  discussed  in  section  8.2. 
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Figure  8.3  Talrik  II™  Sensor  Layout  Diagram  [20] 


8.4.2  Recognition  of  the  Original  Scenarios  with  One  Malfunctioning  Sensor 

To  observe  how  a malfunctioning  sensor  would  affect  the  performance  of  the 
sensory  driver  layer,  one  of  the  sensors  was  disabled  from  getting  any  readings. 
IRDFML,  a “front  left”  sensor  shown  in  Figure  8.3,  was  disabled  and  the  previous 
experiment  described  in  section  8.4.1  was  repeated.  The  complete  results  are  found  in 
Appendix  B and  for  easy  reference,  part  of  the  table  is  given  in  Table  8.3.  The 
experiment  resulted  in  15  misidentifications  out  of  60. 
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Table  8.3  Partial  table  of  modified  Euclidean  distance  of  each  scenario  respect  to  the 
database  of  scenario  templates  with  one  sensor  malfunctioning  from 
Appendix  B 


IlNullTer 

FC 

F 

FRC 

FR 

FLC 

FL 

LC 

L 

RC 

R 

BC 

B 

NullTer 

2 

1270 

245 

1021 

255 

1453 

227 

711 

155 

1295 

190 

736 

38 

FC 

735 

489 

250 

786 

570 

1752 

854 

1414 

880 

1996 

911 

1469 

771 

F 

209 

687 

66 

790 

284 

1444 

386 

918 

362 

1502 

397 

943 

245 

FRC 

991 

1113 

728 

28 

260 

2318 

1198 

1730 

1152 

1248 

831 

1703 

1023 

FR 

306 

1040 

323 

239 

5 

1759 

541 

1045 

467 

1059 

316 

1040 

342 

FLC 

782 

1540 

785 

1661 

1009 

637 

413 

681 

699 

2023 

900 

1516 

818 

FL 

130 

1274 

315 

1155 

399 

967 

123 

539 

203 

1423 

298 

864 

166 

LC 

678 

1916 

921 

1697 

931 

1147 

545 

7 

219 

1971 

836 

1412 

714 

L 

102 

1364 

345 

1121 

355 

1317 

247 

307 

9 

1395 

284 

836 

138 

RC 

1243 

2481 

1486 

1058 

962 

2750 

1490 

1982 

1404 

2 

469 

1977 

1279 

R 

207 

1465 

450 

742 

250 

1714 

454 

946 

368 

470 

5 

941 

243 

BC 

806 

2110 

1063 

1867 

1081 

2313 

1053 

1545 

967 

2099 

996 

4 

502 

B 

36 

1306 

279 

1055 

289 

1543 

283 

775 

197 

1329 

226 

470 

4 

Upon  closer  analysis,  one  can  observe  that  the  15  misidentifications  are  not 
random;  instead,  they  are  related  closely  to  the  correct  scenario.  For  example,  two  of  the 
misidentifications  are  shown  in  Table  8.3.  First,  FC  (object  to  the  close  front)  is 
incorrectly  identified  as  F (object  to  the  front).  Although  the  distance  of  the  object  was 
wrong  (“normal”  distance  rather  than  “close”  distance),  the  sensory  driver  layer  correctly 
identified  the  direction  of  the  object  (“front”).  Complete  analysis  of  Appendix  B shows 
that  most  of  the  errors  occur  while  trying  to  identify  different  types  of  “walls.”  This  is  a 
consequence  of  the  fact  that  as  discussed  in  section  8.2,  “wall”  scenarios  consist  of 
“walls”  shifted  two  inches  from  each  other  and  rotated  15  degrees  from  each  other. 
These  “wall”  scenarios  that  physically  differ  from  each  other  in  small  variations  cause  the 
sensory  driver  layer  to  make  mistakes.  However,  these  mistakes  are  expected  due  to  a 
malfunctioning  sensor.  It  should  be  noted  that  the  mistakes  are  closely  related  to  the 
correct  scenarios,  demonstrating  the  flexibility  of  the  sensory  driver  layer.  Other  errors 
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include  wall  being  mistaken  for  an  object,  wall  being  shifted  at  an  angle,  etc.  But  in  all 
cases,  relative  direction  of  the  obstacle  is  consistent. 


Table  8.4  Partial  table  of  modified  Euclidean  distance  of  each  scenario  respect  to  the 
database  of  scenario  templates  with  two  sensors  malfunctioning  from 
Appendix  B 


NullTer 

FC 

F 

FRC 

FR 

FLC 

FL 

LC 

L 

RC 

R 

BC 

B 

NullTer 

15 

1153 

196 

860 

188 

1430 

232 

724 

168 

1154 

147 

749 

51 

FC 

744 

494 

259 

687 

525 

1761 

863 

1423 

889 

1889 

880 

1478 

780 

F 

242 

668 

75 
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8.4.3  Recognition  of  the  Original  Scenarios  with  Two  Malfunctioning  Sensors 

To  take  the  analysis  further,  a second  sensor  was  disabled  in  addition  to  the  sensor 
disabled  in  section  8.4.2.  The  second  sensor  was  chosen  form  the  same  quadrant  of  the 
first  sensor  and  from  one  of  the  primary  axis  sensors  (sensor  located  in  the  front,  left, 
right,  and  back).  The  second  sensor  chosen  was  the  left  sensor,  IRDL  shown  in  Figure 
8.3.  Again  the  experiment  performed  in  sections  8.4.1  and  8.4.2  was  repeated.  The 
complete  results  are  given  in  Appendix  C with  part  of  the  results  shown  in  Table  8.4. 
With  two  sensors  disabled,  this  experiment  resulted  in  20  errors,  five  more  than  with  just 
one  sensor  disabled.  Analysis  of  the  data  showed  similar  results  with  the  experiment 
with  one  sensor  disabled,  discussed  in  section  8.4.2.  Majority  of  the  misidentifications 
made  errors  in  distance  and  the  “wall”  scenarios.  In  comparing  Table  8.4  (MED  data  of 
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two  sensors  disabled)  with  Table  8.3  (MED  data  of  one  sensor  disabled),  mistakes  were 
made  in  direction  of  the  object  as  well.  This  is  different  from  the  one  sensor  disabled 
case,  in  which  direction  was  pretty  consistent.  Results  in  Appendix  C show  further 
degradation  in  the  performance  of  the  sensory  driver  layer,  due  to  two  sensor 
malfunction. 

8.4.4  Detailed  Sensor  Template  Mapping  for  Objects 

Thus  far  in  the  proceeding  sections,  namely  8.4.1,  8.4.2,  and  8.4.3,  sensory  driver 
layer  was  tested  with  scenarios  identical  to  the  scenarios  used  during  the  learning  process. 
The  next  few  experiments  test  the  sensory  driver  layer  with  scenarios  different  from  the 
scenarios  used  during  the  learning  process.  The  first  of  these  experiments  creates  a 
sensor  template  map  covering  front  half  of  the  AMR.  The  sensor  template  map  was 
created  by  placing  a 5 14  x 5 14  inch  block  in  different  locations  within  a 180  degree 
region  about  the  AMR.  The  block  was  placed  every  inch  starting  from  seven  inches  to  24 
inches  in  a line  pointing  radially  out  from  the  center  of  the  AMR.  This  was  done  for 
every  10  degrees  for  the  180  degree  region  around  the  front  of  the  AMR. 

The  scenario  map  from  the  experiment  is  presented  in  Figure  8.4.  From  this  plot, 
one  can  observe  that  the  region  surrounding  the  original  location  of  a scenario  is  also 
identified  as  the  same  scenario.  For  example,  gray  dots  in  Figure  8.4  represents  “object 
to  the  front”  (F).  Original  location  of  the  F scenario  is  directly  in  front  of  the  AMR  18 
inches  away  from  the  center  of  the  AMR.  Notice  that  many  other  dots  in  the  region  also 
share  the  same  classification  of  F.  These  groupings  of  regions  around  the  original 
location  of  the  scenarios  indicate  that  similar  scenarios  will  be  properly  identified  as 


intended. 
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Figure  8.4  Scenario  mapping  of  a 5 14  x 5 *4  inch  block  in  the  180  degree  region  in 
front  of  the  AMR.  Each  radial  line  represents  10  degree  offsets  and  each 
point  represents  one  inch  offsets 

There  are  some  dots  representing  “wall”  identifications.  They  occur  very  close  to 
the  AMR  or  in  boundaries  between  to  regions.  These  misidentifications  make  sense  in 
that  more  sensors  would  make  higher  intensity  readings  from  an  object  that  is  extremely 
close.  In  the  case  of  boundaries  being  identified  as  “walls,”  it  results  from  the  object 
occupying  parts  of  two  regions,  which  would  again  result  in  multiple  sensors  registering 
reading.  This  situation  again  leads  to  misidentification  as  a “wall.”  Lastly,  one  will 
notice  that  not  all  measurements  extend  out  the  same  distance.  The  lack  of  a plot 
represents  the  distance  at  which  the  sensory  driver  layer  started  identifying  a NullTemp, 
an  empty  environment.  The  differences  in  this  and  in  other  characteristics  are  due  to 
inherent  differences  in  the  sensor  and/or  their  alignment.  The  results  in  Figure  8.4  show 


75 


the  adaptability  of  the  sensory  driver  layer  to  compensate  for  these  slight  differences  in 
the  sensors. 


-10  -50  5 10 

Distance  in  inches 


Figure  8.5  The  plot  of  lengthening  a 5 *4  inch  object  to  a 27  !4  inch  object  in  2 inch 
increments 


8.4.5  Object  Morphing 

The  scenario  mapping  experiment  showed  the  effects  of  varying  the  location  of 
objects  throughout  the  region.  This  section  presents  experiments  that  vary  the  shapes  and 
sizes  of  the  objects.  In  the  first  experiment  a block  (5  XA  x 5 !4  inch),  used  as  an  obstacle 
during  the  learning  procedure,  was  lengthened  by  two  inch  increments  to  observe  at  what 
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length  the  object  would  start  to  be  recognized  as  a “wall.”  Figure  8.5  presents  a graphical 
result  of  the  experiment.  The  objects  were  placed  1 1 inches  from  the  center  of  the  AMR. 
The  object  was  identified  as  an  obstacle  from  5 *4  inches  in  length  toll  *4  inches,  as  a 
“wall  far  away”  from  13  *4  inches  to  19  *4  inches,  and  as  a “wall”  from  21  *4  inches  to  27 
*4  inches.  The  dimensions  of  the  objects  used  during  the  learning  process  for  an  obstacle 
was  5 !4  inches,  while  a wall  was  28  inches  in  length.  Sensory  driver  layer  identified  the 
object  as  a wall  starting  at  13  14  inches.  This  demonstrates  that  the  sensory  driver  layer 
can  deal  with  objects  of  different  shapes  and  sizes.  These  discrepancies  are  resolved  by 
just  searching  for  a closest  scenario. 

8.5  Sensory  Driver  Layer’s  Ability  to  Compensate  for  sensor  malfunctions 

Recall  the  experiments  performed  in  sections  8.4.2  and  8.4.3,  where  one  and  two 
sensors  were  disabled,  respectively.  With  one  sensor  disabled,  it  resulted  in  15  errors  out 
of  60;  while,  with  two  sensors  disabled,  it  resulted  in  20  errors  out  of  the  same  60. 
Normally,  the  problem  would  be  resolved  by  replacing  the  malfunctioning  sensor  or 
sensors  and  recalibrating  the  parameters.  But  what  if  for  some  reason  the  sensors  could 
not  be  replaced.  Then  it  would  require  reprogramming  of  the  AMR.  The  programmer 
has  to  manually  compensate  for  the  malfunctioning  sensors  by  recreating  the  drivers. 
This  would  be  a time  consuming  and  complex  process.  Even  worse,  if  the  code  depends 
on  direct  sensor  values,  much  of  the  code  probably  has  to  be  rewritten  to  compensate  for 
the  bad  sensors.  These  options  would  not  be  acceptable  in  most  situations. 

The  sensory  driver  layer,  presented  by  AEDEC  architecture,  allows  for  quick  and 
easy  method  for  compensating  for  sensor  malfunctions  and/or  changes  by  simply 
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Table  8.5  Partial  table  of  modified  Euclidean  distance  of  each  scenario  respect  to  the 
new  database  of  scenario  templates  with  two  sensors  malfunctioning  from 
Appendix  D 
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relearning  the  scenarios.  In  the  first  experiment,  AMR  with  one  bad  sensor  relearned  the 
60  scenarios.  Then  it  was  tested,  as  described  in  section  8.4.1,  to  see  if  any 
improvements  could  be  made  on  the  15  errors  it  had  before  the  retraining.  It  passed  with 
flying  colors,  improving  from  15  errors  to  none.  It  properly  identified  all  60.  This 
process  was  extended  to  the  case  where  two  sensors  were  bad,  which  previously  resulted 
in  20  out  of  60  errors.  Even  with  two  bad  sensors,  it  properly  identified  all  60  scenarios 
after  repeating  the  learning  process.  Complete  results  are  given  in  Appendix  D. 

8.6  Real-time  Self  Correcting  Sensory  Driver  Layer 

In  section  8.5,  sensory  driver  layer  easily  adapted  to  the  malfunctioning  sensors  by 
relearning  the  sensory  driver  layer  with  the  malfunctioning  sensors.  Although  it  is  a 
simple  and  efficient  method,  it  does  require  a relearning  process.  If  an  AMR  can  detect 
and  compensate  for  a malfunctioning  sensor  in  real-time  it  would  be  that  much  more 
useful.  A closer  comparison  of  the  new  sensory  driver  layer  to  the  original  resulted  in  a 
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surprising  but  an  obvious  result.  The  new  drivers  were  almost  identical  to  the  old,  with 
the  exception  of  the  malfunctioning  sensors.  The  net  effect  of  the  relearning  process  was 
equivalent  to  just  dropping  the  malfunctioning  sensor  from  the  input  sensor  vector  and 
dropping  the  malfunctioning  sensor’s  component  from  each  of  the  previously  learned 
scenarios.  Detection  of  a malfunctioning  sensor  is  surprisingly  easy.  During  the  initial 
boot  up  procedure,  an  AMR  can  make  sensor  readings  and  compare  against  its  scenarios 
and  just  disable  any  sensor  with  improper  readings. 

As  long  as  an  AMR  can  detect  a malfunctioning  sensor,  it  can  ignore  that  sensor 
reading  and  also  ignore  the  MED  generated  by  the  offending  sensor  respect  to  each  of  the 
scenarios.  In  effect,  adapting  the  sensory  driver  layer  in  real-time  to  malfunctioning 
sensor  or  sensors. 


CHAPTER  9 

ACTUATOR  DRIVER  LAYER 

Just  as  the  sensory  driver  layer  can  organize  and  extract  pertinent  information 
from  the  environment,  an  actuator  driver  layer  can  implement  actions  in  a consistent  and 
deliberate  manner.  Actuator  driver  layer  traditionally  in  the  context  of  Autonomous 
Mobile  Robot  (AMR)  have  been  refined  and  calibrated  through  “manual  tweaking”  of 
parameters,  or  worse,  just  ignored  and  covered  up  by  coding  more  complex  and 
sophisticated  programs  to  minimize  the  effect  of  actuator  inconsistencies.  As  discussed 
in  section  6.1,  ignoring  actuator  inconsistencies  can  greatly  affect  the  performance  of  the 
AMR. 

On  the  other  hand,  if  an  effort  is  made  to  account  for  actuator  inconsistencies,  the 
process  of  creating  an  actuator  driver  layer  becomes  a long  and  complex  procedure. 
Furthermore,  this  leads  to  other  problems.  First,  programmer’s  bias  becomes  a factor  in 
the  driver,  which  may  lead  to  undesirable  tendencies  on  an  AMR.  Second,  different 
drivers  usually  lack  consistency  between  them,  due  to  different  programmer  coding  the 
drivers  on  different  AMR.  This  lack  of  consistency  results  in  a high  learning  curve  and 
high  level  code  incompatibilities.  These  issues  bring  forward  similar  problems  discussed 
in  Chapter  8.  However,  actuator  driver  layer  using  the  AEDEC  architecture  addresses 
these  issues.  This  chapter  presents  actuator  driver  layer  and  the  supporting  experiments. 
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9.1  Innate  Knowledge 

AMR  starts  with  certain  innate  knowledge  preprogrammed  for  the  actuator  driver 
layer  learning  procedure.  AMR  knows  that  there  are  two  actuators  that  control 
movement  and  the  AMR  knows  how  to  access  the  two  actuators  (motors).  The  speed 
values  of  fast  average,  slow,  and  very  slow  are  preprogrammed.  However,  the  AMR 
does  not  know  which  actuator  is  the  left  or  the  right  and  which  direction  is  forward  or 
backwards.  Finally,  the  AMR  starts  off  with  innate  knowledge  about  the  environmental 
characteristics  and  the  behavior  to  be  executed.  With  this  limited  set  of  simple  innate 
knowledge  (explained  further  in  the  following  sections),  actuator  driver  layer  was  learned 
using  the  AEDEC  architecture. 

9.2  Autonomous  Configuration  of  the  Actuators 

The  algorithm  presented  in  this  section  is  a refined  version  of  the  algorithm  found 
in  section  5.2.5.  The  previous  algorithm  did  not  have  the  benefit  of  a sensory  driver 
layer.  Hence  the  new  algorithm  taking  advantage  of  the  sensory  driver  layer  is  more 
elegant  than  its  predecessor.  Previous  algorithm  depended  on  innate  knowledge  about 
the  fact  that  there  are  more  primary  sensors  directed  in  the  forward  direction  than  in  the 
back.  The  algorithm  presented  in  this  section  simply  gets  the  definition  from  the  sensory 
driver  layer.  Both  algorithms  use  a simple  enclosed  environment.  For  example,  an 
environment  made  up  of  four  outer  walls  in  the  shape  of  a square. 

The  flowchart  of  the  algorithm  is  given  in  Figure  9.1.  It  takes  advantage  of  the 
simple  motor  properties,  in  that  there  are  only  four  possible  motor  direction 
combinations,  as  shown  in  Table  9.1.  If  the  AMR  assumes  that  the  two  actuators  are 
configured  correctly  and  go  forward,  there  are  only  three  possible  outcomes:  move 
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forward  and  detect  a wall  in  the  front,  move  backward  and  detect  a wall  in  the  back,  or 
turn  in  place  and  detect  no  wall.  For  the  last  case  where  the  AMR  did  not  detect  a wall, 
correction  is  made  by  changing  the  direction  of  just  one  actuator,  since  turning  in  place  is 
a result  of  one  actuator  direction  being  correct  and  the  other  being  incorrect.  The  change 
in  direction  of  just  one  actuator  would  result  in  both  actuator  directions  being  correct  or 
both  being  wrong.  Regardless  of  the  result,  go  forward  command  would  now  result  in  a 
wall  being  detected  either  in  the  front  or  in  the  back.  If  a wall  was  detected  in  the  back, 
both  actuator  directions  are  changed  resulting  in  both  motor  directions  being  correct. 
This  algorithm  guarantees  that  ultimately  the  AMR  will  get  to  a state  where  both  actuator 
directions  are  correct  and  the  AMR  is  facing  a wall  in  the  front. 


Table  9.1  Four  possible  combinations  of  the  motor  directions. 


Left  Motor  Direction 

Right  Motor  Direction 

Result  of  Going  Forward 

Case  1 

CORRECT 

CORRECT 

MOVE  FORWARD 

Case  2 

CORRECT 

BACKWARDS 

TURN  IN  PLACE  (right) 

Case  3 

BACKWARDS 

CORRECT 

TURN  IN  PLACE  (left) 

Case  4 

BACKWARDS 

BACKWARDS 

MOVE  BACKWARD 

Next,  the  configuration  of  the  actuator  identifications,  namely  left  and  right,  can 
be  configured  by  a simple  turning  behavior.  A right  turn  by  the  AMR  is  implemented  by 
turning  the  left  actuator  forward  and  the  right  actuator  backward.  A right  turn  will  either 
result  in  a right  turn  if  the  actuator  identifications  are  correct  or  in  a left  turn  if  the 
identifications  are  switched.  Consequently,  if  the  AMR  detects  a wall  to  the  left  before 
the  right  wall,  then  the  actuator  identifications  are  correct  and  if  the  AMR  detects  a wall 
to  the  right  before  the  left  wall,  then  the  actuator  identifications  need  to  be  switched.  By 
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implementing  the  algorithm  presented  in  Figure  9.1,  the  drive  actuators’  polarities  and 
identifications  can  be  autonomously  configured.  Autonomous  configuration  of  the 
actuators  allows  detection  and  correction  of  discrete  production  errors  such  as  incorrect 
actuator  connection. 


Figure  9. 1 Refined  Innate  Learning  algorithm  for  motor  configuration 

9.3  Learning  Algorithm  for  the  Actuator  Driver  Layer 

The  algorithm  presented  is  a refined  version  of  the  algorithm  presented  in  Chapter 
6 titled  Environmental  Reinforcement  Learning.  Instead  of  restating  the  algorithm 
presented  previously  in  Chapter  6,  only  special  properties  and  differences  will  be 
highlighted  in  this  chapter.  For  more  detailed  explanation  of  the  algorithm,  please  refer 
back  to  Chapter  6. 
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9.3.1  Assumptions 

Following  assumptions  are  taken  for  the  learning  of  the  actuator  driver  layer  using 
AEDEC  architecture.  First,  the  variance  of  a behavior  is  constant  (time  invariant)  for  a 
given  state  of  the  AMR.  Second,  the  variance  of  a behavior  depends  only  on  the 
observable  states  of  the  AMR.  Third,  a simple  environment  can  be  configured  to  force 
the  emergence  of  the  desired  behavior.  Fourth,  environmental  reinforcement  (reward  and 
punishment)  is  available  to  the  AMR  as  feedback.  Finally,  it  is  assumed  that  the  sensory 
driver  layer  has  already  been  learned  by  the  AMRs. 

9.3.2  The  Highly  Structured  Environment 

The  learning  environment  is  a 30  foot  hallway.  The  space  between  the  two  walls 
is  two  feet.  In  essence,  the  environment  is  a 30  X 2 foot  rectangular  enclosure.  All  the 
walls  in  the  environment  were  painted  white.  Within  this  simple  yet  highly  effective 
environment,  multiple  primitive  behaviors  can  be  learned  and  refined.  First,  the  straight 
property  of  the  hallway  is  used  to  calibrate  mismatched  drive  actuators.  Second,  the 
known  length  of  the  hallway  is  used  to  assign  distance  metrics  to  the  actuator  driver  layer 
(i.e.,  three  feet).  Third,  the  end  of  the  hallway  is  used  to  learn  absolute  turning  behaviors 
(i.e.,  turning  exactly  180  degrees,  90  degrees,  45  degrees,  etc.). 

9.3.3  Modifications  to  the  Autonomous  Mobile  Robots 

Talrik  II™  and  Mantaray,  described  in  detail  in  Chapter  4,  were  modified  to 
greatly  reduce  the  number  of  learning  trials  necessary  for  the  actuator  driver  layer. 
Previous  experiments,  described  in  Chapter  6,  required  over  3000  iterations  to  complete 
one  phase  of  the  learning.  Over  3000  iterations  were  required  due  to  the  effect  of  battery 
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voltage  on  the  actuator  driver  layer.  For  the  same  motor  speed  command,  the  actual 
speed  of  the  motor  changed  corresponding  to  changes  in  the  battery  voltage.  Instead  of 
creating  one  actuator  driver  layer,  the  learning  algorithm  had  to  create  17  different 
versions  (each  corresponding  to  different  battery  voltage)  of  the  actuator  driver  layer.  To 
minimize  this  complexity,  battery  voltage  used  by  the  actuators  was  regulated  to  output 
constant  eight  volts.  This  was  implemented  using  three  DC  to  DC  voltage  regulators 
working  in  parallel.  Three  was  used  to  satisfy  the  peak  current  requirements  of  the 
actuators.  Also,  casters  on  both  AMRs  were  changed  to  static  skid  casters  (no  moving 
parts). 


9.3.4  Incorporation  of  the  Sensory  Driver  Layer 

Sensory  driver  layer  provides  valuable  information  that  can  be  used  to  enhance 
learning  while  still  maintaining  independence  from  the  programmer  (i.e.,  minimizing  bias 
and  the  need  for  supervision).  Certain  sensor  characteristics  are  extracted  from  the 
scenario  templates  created  in  Chapter  8.  The  sensor  most  affected  by  objects  directly  in 
the  front  of  the  AMR  is  identified  by  looking  for  the  sensor  registering  the  biggest  change 
in  the  F (obstacle  to  the  Front)  scenario.  Next  sensor  identified  is  a sensor  most  affected 
by  L (obstacle  to  the  Left)  and  FL  (obstacle  to  the  Front  Left)  scenarios  as  defined  by  the 
sensory  driver  layer.  Last  sensor  identified  is  a sensor  most  affected  by  R (obstacle  to  the 
Right)  and  FR  (obstacle  to  the  Front  Right)  scenarios.  Three  sensors  are  identified  in  this 
procedure,  front  sensor,  left  angle  sensor  (sensor  to  the  left  with  slight  bias  toward  the 
front),  and  right  angle  sensor  (sensor  to  the  right  with  slight  bias  toward  the  front). 

Once  identified,  their  threshold  characteristics  are  extracted  from  the 
corresponding  scenarios.  Front  sensor’s  high  threshold  is  the  value  of  the  sensor  in  the 


85 


FC  (obstacle  to  the  Front  Close)  scenario  template,  while  low  threshold  is  from  the  F 
(obstacle  to  the  Front)  scenario.  Right  angle  sensor’s  thresholds  are  high,  medium,  low 
each  extracted  from  WRC  (Wall  to  the  Right  Close),  WR  (Wall  to  the  Right),  WRF  (Wall 
to  the  Right  Far),  respectively.  Similarly,  left  angle  sensor’s  thresholds  are  high, 
medium,  low  each  extracted  from  WLC  (Wall  to  the  Left  Close),  WL  (Wall  to  the  Left), 
WLF  (Wall  to  the  Left  Far),  respectively.  By  using  the  sensory  driver  layer  to  pick  the 
sensors  and  their  thresholds,  actuator  driver  layer  has  eliminated  the  dependence  on  a 
human  programmer,  which  was  not  the  case  in  the  previous  algorithm  presented  in 
Chapter  6. 


Figure  9.2  Environment  used  for  calibration  of  mismatched  actuators 

9.3.5  Algorithm  for  Learning  to  Calibrate  Mismatched  Actuators 

Unless  high  precision  actuators  are  used,  actuators  tend  to  have  different 
characteristics,  which  lead  to  different  speeds  for  the  left  and  right  motors.  The  algorithm 
presented  in  this  section  will  autonomously  calibrate  the  mismatched  actuators.  Learning 
depends  on  the  hallway  to  force  a straight  line  behavior.  When  in  the  Safe  Center  zone, 
as  shown  in  Figure  9.2,  the  AMR  assumes  the  current  speed  parameters  are  correct  and 
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tries  to  go  forward.  If  the  right  motor  is  faster  than  the  left,  the  AMR  will  arc  to  the  left. 
This  will  cause  the  AMR  to  enter  the  Hit  Left  zone,  which  counts  as  a hit  on  the  left. 
Once  the  hit  occurs,  AMR  tries  to  center  itself.  Once  centered,  the  process  repeats  until 
the  end  of  the  hallway  is  reached. 

Traveling  the  length  of  a 30  foot  hallway,  the  AMR  will  accumulate  hits  on  the 
left  and  hit  on  the  right.  If  one  motor  is  faster  than  the  other,  there  will  be  more  hits  on 
the  opposite  side  of  the  faster  motor.  Once  the  end  of  the  hallway  is  reached,  the  right 
motor  speed  parameter  is  increased  or  decreased,  depending  on  the  respective  hit  counts. 
For  15  seconds  at  the  beginning,  hits  are  ignored  due  to  the  harmful  effect  of  the  turn  180 
degrees  behavior.  During  this  time  the  AMR  focuses  on  centering  itself  in  the  hallway. 
Initially,  the  offset  value  starts  at  4%  of  the  speed  magnitude.  But  as  oscillatory 
conditions  are  encountered,  offset  is  reduced  by  half,  leading  to  finer  and  finer 
adjustments.  The  ending  condition  of  the  learning  algorithm  is  encountered  when  the 
offset  reaches  0. 

The  plot  of  right  motor  speed  values  is  given  in  Figure  9.3.  It  shows  that  by  the 
45th  iteration,  the  learning  was  completed.  The  final  right  motor  speed  value  ended  up 
being  2240,  a correction  of  260  from  the  initial  value  of  2500. 

9.3.6  Algorithm  for  Learning  the  Distance  Metrics 

Distance  metrics  was  implemented  with  a running  average  of  the  time  required  to 
travel  one  foot.  Since  it  is  difficult  to  measure  one  foot  of  travel  accurately,  the  data  was 
not  collected  for  one  foot  increments;  instead,  the  total  distance  of  the  hallway  was  used. 
Due  to  sensor  inaccuracies  and  delay  caused  by  the  software  (up  to  250  milliseconds),  the 
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Iteration  Number 

Figure  9.3  Plot  of  the  right  motor  speed  values 

actual  distance  traveled  varied  from  one  iteration  to  another.  But  the  effects  of  these 
inaccuracies  were  minimized  by  measuring  for  the  total  length  of  the  hallway  and 
dividing  by  the  number  of  feet  traveled.  This  in  effect  divides  the  error  by  approximately 
28.5  (total  number  of  feet  traveled).  Furthermore,  a running  average  was  kept  to  further 
filter  out  the  error.  The  effects  of  these  minimizations  are  evident  in  Figure  9.4.  The 
running  average  for  traveling  one  foot  settles  down  to  2169  milliseconds  by  the  49th 


iteration. 
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Iteration  Number 


Figure  9.4  Plot  of  time  needed  to  travel  one  foot 


9.3.7  Algorithm  for  Learning  the  180  Degree  Turn 

The  ends  of  the  environment,  described  in  section  9.3.2,  are  used  to  implement 
the  learning  of  turning  180  degrees.  When  the  AMR  reaches  the  end  of  the  corridor,  it 
ends  up  parallel  to  the  hallway  facing  the  enclosed  end,  as  shown  in  Figure  9.5  (a).  This 
forces  a 180  degree  turn  by  the  AMR  in  order  to  face  the  open  end  of  the  hallway.  Once 
the  AMR  turns  180  degrees,  it  will  end  up  perfectly  parallel  (no  correction  needed)  or  it 
will  end  up  in  a position  shown  by  Figure  9.5  (c),  representing  an  under  turn,  or  Figure 
9.5  (e),  representing  an  over  turn.  Finally,  AMR  goes  forward  until  left  hit,  shown  in 
Figure  9.5  (f),  or  right  hit,  shown  in  Figure  9.5  (d),  is  detected.  Very  small  errors  in  the 
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(d)  Right  First  Hit  (e)  (f)  Left  First  Hit 


Figure  9.5  Part  of  the  environment  used  for  learning  to  turn  180  degrees 

turning  angle  can  be  detected  by  this  forward  movement.  It  is  analogous  to  drawing  an 
imaginary  line  through  the  axis  of  the  AMR  and  observing  if  the  line  intersects  the  walls 
of  the  hallway.  Only  a perfect  turn  will  result  in  a parallel  path  respect  to  the  hallway.  In 
this  algorithm,  the  line  was  only  extended  for  10  seconds  (the  distance  traveled  by  the 
AMR  in  10  seconds).  Depending  on  whether  left  hit  or  right  hit  occurred,  the  time  to  turn 
180  degrees  is  appropriately  decreased  or  increased. 

The  very  first  turn  time  is  initialized  by  turning  until  the  front  sensor  value  falls 
below  the  low  threshold,  defined  in  section  9.3.4.  For  the  following  iterations,  the 
previous  turn  time  is  executed.  After  every  iteration,  the  turn  time  is  appropriately 
increased  or  decreased  by  the  offset.  Offset  value  is  initialized  as  2.5%  of  the  initial  turn 
time.  But  as  oscillatory  conditions  are  encountered,  offset  is  reduced  by  half,  leading  to 
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finer  and  finer  adjustments.  The  ending  condition  of  the  learning  algorithm  is 
encountered  when  the  offset  reaches  0. 


Trial  Number 

Figure  9.6  Plot  of  time  needed  to  turn  180  degrees 

The  plot  of  the  times  needed  to  turn  180  degrees  is  given  in  Figure  9.6.  It  shows 
that  by  the  44th  iteration,  the  learning  was  completed.  The  final  turn  time  ended  up  being 
3602  milliseconds. 

9.3.8  Qualitative  Analysis  of  the  Actuator  Driver  Layer 

Previous  sections  presented  quantitative  analysis  of  the  actuator  driver  layer 
through  the  graphs  of  the  learning  convergence  of  the  three  refined  primitive  behavior 
parameters,  namely  calibration  of  the  left  and  right  motor  speeds,  turning  180  degrees, 
and  traveling  units  of  distance.  This  section  presents  the  three  behaviors’  performance  in 
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a qualitative  manner.  This  is  accomplished  by  tracing  the  AMR’s  path  using  the 
Autonomous  Mobile  Robot  Behavior  Analysis  Tool  (AMRBA). 

In  the  past,  a time  elapsed  exposure  camera  was  used  to  record  the  path  of  an  AMR 
for  presentation  and  analysis.  It  was  a cumbersome  process.  The  AMR  would  carry  a 
small  LED  light  source,  which  in  turn  would  show  up  in  the  film  as  streaks  of  light 
representing  the  path  of  the  AMR.  It  is  similar  to  the  streaking  of  the  automobile 
headlights  in  time  elapsed  photo.  Consequently,  a new  analysis  tool,  AMRBA,  was 
developed. 

AMRBA  is  an  adaptation  of  the  image  processing  tool  developed  in  the  Machine 
Intelligence  Laboratory  at  the  University  of  Florida.  It  is  capable  of  capturing  and 
processing  video  streams  in  real-time  for  various  image  processing  applications. 
AMRBA  uses  a partial  set  of  functions  to  implement  an  analysis  tool  for  AMR  behavior 
analysis. 

The  AMR  is  programmed  to  go  forward  five  feet,  turn  180  degrees,  and  come  back 
five  feet.  This  simple  behavior  was  initially  performed  without  the  benefit  of  the  learned 
behaviors  and  then  a second  time  with  the  learned  behaviors.  However,  in  both  of  the 
instances,  the  180  degree  turn  used  the  learned  behavior,  allowing  for  comparison  of  the 
two  paths  of  the  AMR.  Figure  9.7  (a)  shows  the  path  of  the  AMR  performing  the  given 
behavior  without  the  benefit  of  the  learning,  while,  Figure  9.7  (b)  shows  the  path  of  the 
AMR  after  learning.  From  the  comparison  of  the  two  paths,  benefits  of  learning  become 
obvious.  Stable  and  predictable  actuator  driver  layer  will  result  in  a more  robust  system. 


92 


Figure  9.7  The  path  of  the  AMR  before  learning  (a)  and  after  learning  (b) 


CHAPTER  10 

ANALYSIS  OF  THE  AEDEC  SYSTEM 


The  final  set  of  experiments  performed  in  support  of  the  AEDEC  system  are 
presented  and  analyzed  in  this  chapter.  This  chapter  emphasizes  the  AEDEC 
architecture’s  ability  to  simplify  programming  of  behaviors  and  its  ability  to  have 
behavioral  code  that  is  both  portable  and  functional  on  differing  AMR  platforms.  Two 
behavioral  algorithms  were  coded  using  the  sensory  and  actuator  drivers  autonomously 
created  using  the  AEDEC  architecture.  The  first  behavior  is  the  classic  “obstacle 
avoidance”  and  the  second  behavior  is  a right  wall  following  behavior. 

In  analyzing  the  code  for  each  of  the  behavior,  one  should  note  the  complete 
absence  of  parameters  and  any  specific  reference  to  sensors  or  actuators.  This  high  level 
abstraction  by  the  AEDEC  architecture  results  in  a very  simple  code,  where  one  can 
program  an  AMR  without  understanding  any  specific  properties  of  the  AMR.  Secondly, 
AEDEC’ s abstraction  allows  code  portability  among  different  AMRs,  since  code  is 
written  to  the  drivers’  specifications  rather  than  the  individual  specifications  of  the 
AMRs. 

Instead  of  requiring  an  intimate  knowledge  of  the  AMR,  one  can  program  using 
high  level  abstractions.  For  example,  programming  an  AMR  trained  under  the  AEDEC 
architecture  requires  a familiarity  of  the  60  scenarios  presented  in  Section  8.2  and  the 
eight  actuator  control  functions,  which  is  far  simpler  than  trying  to  figure  out  the  effect  of 
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each  sensor’s  orientation,  effect  of  each  sensor’s  characterizations,  effect  of  motor 

commands,  etc. 

void  obstacleAvoidance() 

{ 

#define  SenarioListSizeForOA  8 
enum  sensorTemplateType  senarioMatch; 
enum  sensorTemplateType 

senarioList[SenarioListSizeForOA] 

= {NullTemp,  FC,  F,  FRC,  FR,  FLC,  FL,  WFF};  /*  Main  list  7 


while(l) 

{ 

readSensors(AlllrSensorMode); 

senarioMatch 

= findMatchFromListOfSenTemp(SenarioListSizeForOA,senarioList); 

switch(senarioMatch) 

{ 

case  NullTemp: 

Forward(); 

break; 

case  FC:  case  FRC:  case  FLC:  case  WFF: 

TurnRight(); 
break; 
case  F: 

PivotTurnRight(); 
break; 
case  FR: 

ArcLeftQ; 
break; 
case  FL: 

ArcRight(); 

break; 

} 

} 

} 

Figure  10.1  Obstacle  avoidance  code 


10.1  Obstacle  Avoidance  Behavior 

As  shown  in  Figure  10.1,  behavior  coding  turns  into  a simple  categorization 
problem.  First,  make  a list  of  scenarios  that  one  needs  for  the  behavior.  For  example,  for 
obstacle  avoidance  you  would  need  scenarios  that  represent  obstacles  to  the  front  of  the 
AMR.  Consequently,  the  “main  list,”  shown  in  Figure  10.1,  consists  of  seven  scenarios 
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representing  different  types  of  obstacles  to  the  front  plus  the  NullTemp  representing  no 
obstructions.  The  main  list  is  part  of  just  one  possible  solution  and  is  no  way  a unique 
solution.  One  could  have  made  the  main  list  with  just  four  elements  and  still  implement 
obstacle  avoidance.  By  having  more  scenarios  in  you  main  list,  programmer  has  finer 
control  of  the  behavior.  Final  task  of  programming  is  to  take  the  main  list  and  associate 


(c)  (d) 

Figure  10.2  The  path  of  the  obstacle  avoidance  program  for  Mantaray  (a)  and  (c)  and 
for  Talrik  II™  (b)  and  (d) 

them  with  the  proper  actuator  commands.  For  example,  Forward()  action  is  triggered 
when  NullTemp  (open  space)  is  the  closest  match. 
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The  obstacle  avoidance  code  makes  a sensor  reading  and  the  closest  matching 
scenario  from  the  main  list  is  picked.  Finally,  the  associated  task  for  that  scenario  is 
executed  and  then  the  process  repeats.  Without  being  hindered  by  technical  detail,  the 
obstacle  avoidance  code  took  less  than  ten  minutes  to  implement.  The  code  is 
independent  of  the  AMR  and  is  only  dependent  on  the  accuracy  of  the  sensory  and 
actuator  driver  layers.  As  shown  in  Figure  10.2,  the  same  obstacle  avoidance  code 
worked  for  different  environment  and  more  importantly  for  different  AMRs,  namely 
Talrik  II™  and  Mantaray. 

10.2  Wall  following  behavior 

Next  a more  complex  behavior  was  implemented.  Wall  following  behavior  code 
uses  26  scenarios  in  the  main  list  as  shown  in  Figure  10.3.  The  main  list  is  primarily 
made  up  of  “right  wall”  scenarios  (refer  back  to  section  8.2  for  scenario  definitions)  and 
“corner”  scenarios.  Again,  the  structure  of  the  code  is  the  same  as  the  obstacle  avoidance 
code.  Populate  the  main  list  with  the  scenarios  needed  for  the  wall  following  behavior 
and  associate  each  scenario  with  the  proper  actuation.  Most  of  the  scenarios  in  the  main 
list  are  obvious  choices  except  for  RC  (obstacle  close  to  the  right)  and  BRC  (obstacle 
close  to  the  back  right).  These  two  scenarios  were  required  for  maneuvering  around  very 
sharp  comers.  Without  these  two  scenarios  the  AMRs  would  get  dangerously  close  to  the 
comers,  resulting  in  sharp  jerky  movements.  If  one  can  imagine  a very  sharp  comer  to 
the  right  of  the  AMR,  the  comer  would  look  similar  to  the  RC  and  BRC  scenarios. 

The  wall  following  behavior  code  took  about  20  minutes  to  implement.  The  bulk 
of  the  time  was  spent  in  analysis  and  correction  of  the  sharp  comers.  The  coding  process 
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void  wallFollowingRight() 

{ 

#define  SenarioListSizeForWF  26 
enum  sensorTemplateType  senarioMatch; 
enum  sensorTemplateType 

senarioList[SenarioListSizeForWF] 

= {NullTemp,  WRC,  WR,  WRF,  WFR15C,  WFR15,  WFR15F,  WFR30C, 
WFR30,  WFR30F,  WBR15C,  WBR15,  WBR15F,  WBR30C, 

WBR30,  WBR30F,  WF,  CF,  CFR,  CR,  CBR,  CB,  DF,  DB,  RC,  BRC}; 

while(l) 

{ 

readSensors(AlllrSensorMode); 

senarioMatch  = findMatchFromUstOfSenTemp(Senariol_istSizeForWF,senarioList); 

switch(senarioMatch) 

{ 

case  WR:  case  CBR:  case  DB: 

Forward(); 

break; 

case  CF:  case  CFR:  case  CR:  case  DF:  case  WFR15C: 

case  WFR30C:  case  WFR30:  case  WF: 

TurnLeft(); 

break; 

case  CB:  case  WBR30C:  case  WBR30:  case  WBR30F:  case  NullTemp: 
PivotTurnRight(); 
break; 

case  WRC:  case  WFR15F: 

ArcLeft(); 

break; 

case  WRF:  case  WBR15C:  case  WBR15:  case  WBR15F:  case  RC:  case  BRC: 
ArcRight(); 
break; 

case  WFR30F:  case  WFR15: 

PivotTurnLeft(); 

break; 

} 

} 

} 

Figure  10.3  Wall  following  code  (right) 


was  very  different  from  traditional  coding  experiences.  On  does  not  have  to  think 
about  sensors’  interaction  with  the  environment,  the  proper  sensor  thresholds,  or  even 
how  to  control  the  AMR.  Instead  the  programming  focused  on  high  level  abstracted 
knowledge  about  the  environment  and  about  the  actions  that  the  AMR  is  expected  to 
perform. 
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Once  the  code  worked  on  one  AMR,  the  same  exact  code  was  downloaded  to  the 
second  AMR.  The  same  wall  following  code  worked  just  as  well  on  the  second  AMR  as 
it  did  on  the  first  AMM  without  any  modification.  Even  with  this  complex  and  behavior 
specific  code,  the  same  wall  following  program  worked  on  both  of  the  AMRs.  The  paths 
of  the  wall  following  behavior  for  the  two  AMRs  are  presented  in  Figure  10.4. 


(a)  (b) 

Figure  10.4  The  path  of  the  wall  following  program  for  Talrik  II™  (a)  and  for 
Mantaray  (b) 


CHAPTER  1 1 
CONCLUSION 


Autonomous  Evolution  of  sensory  and  actuator  Driver  layers  through 
Environmental  Constraints  (AEDEC)  presents  a novel  methodology  by  which  the  sensory 
and  actuator  drivers  are  created  autonomously  by  the  AMR.  This  greatly  alleviates  the 
need  of  the  programmer  to  code  the  drivers  and  to  characterize  and  document  each  sensor 
and  each  actuator.  AEDEC  architecture  creates  a robust  system  in  that  it  also  allows  for 
discrete  production  errors  detection  through  innate  knowledge  about  the  environment. 
The  AMR  using  AEDEC  can  reconfigure  and  compensate  for  errors  in  hardware  wiring 
and  sensor  malfunctions. 

Coding  for  the  AMR  is  simplified  greatly.  Instead  of  coding  for  a specific  AMR, 
one  codes  for  the  drivers  that  the  AMR  has  learned.  This  greatly  simplifies  the  coding 
process.  One  does  not  need  an  understanding  of  the  AMR’s  sensors  or  actuator  to  start 
coding  for  the  AMR.  One  does  not  need  any  knowledge  about  the  sensors’  thresholds  or 
their  specific  characterization  or  even  how  the  actuators  are  controlled.  Since  the 
consistency  of  the  drivers  between  different  AMRs  are  guaranteed  by  the  highly 
structured  environment,  a behavioral  code  written  for  the  AEDEC  architecture  will  work 
on  other  AMRs  using  the  AEDEC  drivers,  resulting  in  high  level  behavior  code 
portability. 

The  realization  of  AEDEC  architecture  in  conjunction  with  Innate  Learning  (IL) 
and  Environmental  Reinforcement  Learning  (ERL)  provides  invaluable  insight  into 
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discrete  production  error  detection  and  correction,  self-calibration,  reliability,  cost 
reduction,  self-programming,  uniform  command  structure,  and  code  portability. 


APPENDIX  A 

TABLE  OF  MODIFIED  EUCLIDEAN  DISTANCE  OF  EACH  SCENARIO 
RESPECT  TO  THE  DATABASE  OF  SCENARIO  TEMPLATES 
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3740 

WRF 

484 

418 

3475 

3110 

2641 

3690 

3370 

2906 

3679 

3456 

WFR15C 

1041 

1017 

3952 

3661 

3254 

3909 

3711 

3363 

3724 

3593 

WFR15 

937 

877 

3652 

3335 

2906 

3683 

3447 

3061 

3552 

3391 

WFR15F 

876 

758 

3317 

2980 

2537 

3428 

3156 

2738 

3347 

3160 

WFR30C 

1445 

1367 

3764 

3519 

3150 

3553 

3419 

3143 

3294 

3201 

WFR30 

1411 

1301 

3574 

3305 

2916 

3433 

3269 

2961 

3206 

3095 

WFR30F 

1291 

1151 

3346 

3053 

2640 

3283 

3085 

2739 

3102 

2967 

WBR15C 

240 

384 

4149 

3802 

3355 

4360 

4050 

3600 

4345 

4128 

WBR15 

198 

258 

3753 

3400 

2949 

3958 

3640 

3190 

3953 

3728 

WBR15F 

238 

182 

3219 

2866 

2415 

3428 

3106 

2656 

3431 

3202 

WBR30C 

87 

275 

4208 

3857 

3408 

4463 

4133 

3675 

4484 

4249 

WBR30 

30 

92 

3631 

3274 

2821 

3870 

3536 

3078 

3893 

3654 

WBR30F 

80 

18 

3125 

2768 

2315 

3364 

3028 

2570 

3391 

3150 

WLC 

3621 

3055 

28 

73 

196 

125 

95 

115 

310 

293 

WL 

3284 

2718 

45 

20 

73 

198 

108 

62 

393 

336 

"whf 

2888 

2322 

157 

56 

13 

374 

228 

98 

577 

484 

WFL15C 

3850 

3292 

215 

284 

429 

CM 

CM 

58 

168 

67 

84 

WFL15 

3502 

2936 

147 

168 

261 

46 

18 

56 

117 

86 

WFL15F 

3154 

2596 

153 

122 

157 

138 

48 

14 

225 

150 

WFL30C 

3867 

3309 

466 

539 

666 

117 

169 

291 

28 

47 

WFL30 

3550 

2992 

423 

454 

535 

124 

122 

188 

49 

WFL30F 

3316 

2750 

407 

398 

439 

180 

122 

130 

119 

“52 

WBL15C 

3649 

3073 

152 

181 

274 

561 

471 

395 

858 

819 

WBL15 

3134 

2554 

187 

124 

105 

584 

452 

308 

867 

802 

WBL15F 

2752 

2168 

355 

210 

85 

730 

558 

348 

989 

896 

WBL30C 

3340 

2808 

631 

600 

573 

1226 

1098 

910 

1555 

1510 

WBL30 

2968 

2418 

683 

582 

459 

1246 

1098 

858 

1565 

1508 

WBL30F 

2661 

2085 

790 

623 

418 

1321 

1137 

845 

1624 

1537 

WFC 

3462 

3086 

2043 

2042 

2035 

1500 

1506 

1532 

1157 

1124 

WF 

3296 

2872 

2009 

1988 

1931 

1468 

1472 

1468 

1129 

1094 

~WfT~ 

3103 

2635 

1954 

1905 

1798 

1449 

1437 

1393 

1128 

1083 

vr~ 

3453 

3283 

2408 

2431 

2476 

1877 

1895 

1957 

1554 

1533 

CFR 

2117 

2117 

3636 

3603 

3516 

3093 

3099 

3077 

2750 

2717 

CFL 

4500 

4038 

1031 

1146 

1319 

642 

782 

964 

527 

628 

7JR- 

961 

1193 

5072 

4799 

4388 

4905 

4761 

4457 

4640 

4547 

~CT~ 

4707 

4161 

572 

807 

1072 

507 

649 

825 

562 

671 

CbR 

954 

1214 

4545 

4338 

4005 

5088 

4854 

4436 

5225 

5090 

CBL 

3317 

3089 

1380 

1447 

1562 

1893 

1809 

1727 

2198 

2165 

1708 

1688 

3121 

2996 

2779 

3822 

3666 

3342 

4169 

4114 

DF 

3319 

3329 

2612 

2731 

2910 

2227 

2375 

2563 

2118 

2227 

DB 

2579 

2849 

3136 

3211 

3334 

3617 

3557 

3483 

3884 

3871 

106 


WFL30F 

WBL15C  | WBL15 

'VVBL15F  IWBL30C 

WBL30 

W6L30F 

WFC TWF 

"WFF — 

Null  lemp 

1653 

TTBR' 

— 

908 

TTTT 

1249 

845 

2625 

— 2057 

1620 

TC- 

1303 

2585 

2172 

1898 

2740 

2321 

1963 

1003 

599 

330 

F 

1218 

1896 

1439 

1121 

1941 

1500 

1120 

1612 

1138 

761 

FRC 

2481 

2765 

2264 

1904 

2710 

2245 

1841 

1573 

1275 

1084 

TR- 

1890 

2052 

1551 

1191 

1997 

1532 

1128 

1892 

1472 

1151 

FLC 

249 

1481 

1380 

1372 

2166 

1957 

1809 

1065 

911 

764 

FL 

733 

1377 

1056 

854 

1662 

1309 

1017 

1633 

1257 

932 

TC 

753 

493 

226 

124 

748 

477 

327 

2377 

2099 

1796 

L 

1248 

1168 

727 

445 

1181 

774 

452 

2474 

2016 

1601 

"RC- 

2998 

3056 

2551 

2187 

2991 

2524 

2118 

2980 

2728 

2441 

"R 

1853 

1969 

1468 

1108 

1914 

1449 

1045 

2449 

2023 

1624 

"bc- 

2516 

2022 

1733 

1465 

1561 

1382 

1240 

3476 

2940 

2475 

"B- 

1688 

1746 

1269 

917 

1629 

1204 

838 

2660 

2122 

1655 

BRC 

2582 

2658 

2157 

1793 

2499 

2080 

1722 

3506 

3014 

2545 

"SR- 

1764 

1854 

1353 

993 

1775 

1322 

930 

2732 

2212 

1745 

BLC 

1986 

864 

611 

475 

561 

344 

242 

3354 

2816 

2349 

BL 

1606 

1382 

917 

601 

1277 

850 

512 

2728 

2188 

1717 

WRC 

3777 

4001 

3500 

3140 

3894 

3455 

3077 

2849 

2681 

2500 

WR 

3422 

3614 

3113 

2753 

3527 

3078 

2690 

2688 

2500 

2297 

WRF 

3122 

3250 

2745 

2381 

3165 

2708 

2312 

2650 

2424 

2183 

WFR15C 

3359 

3833 

3372 

3046 

3800 

3375 

3009 

2097 

1979 

1854 

WFR15 

3127 

3521 

3036 

2692 

3484 

3037 

2651 

2051 

1895 

1744 

WFR15F 

2868 

3146 

2657 

2309 

3107 

2654 

2262 

2088 

1894 

1703 

WFR30C 

3015 

3783 

3326 

3008 

3814 

3379 

3005 

1589 

1479 

1384 

WFR30 

2887 

3537 

3072 

2746 

3552 

3111 

2731 

1577 

1445 

1332 

WFR30F 

2731 

3261 

2780 

2440 

3246 

2795 

2405 

1629 

1455 

1306 

WBR15C 

3804 

3928 

3455 

3115 

3791 

3394 

3056 

3196 

3024 

2821 

WBR15 

3398 

3558 

3069 

2721 

3463 

3038 

2674 

3010 

2812 

2579 

WBR15F 

2868 

3024 

2535 

2187 

2949 

2514 

2140 

2726 

2482 

2211 

WBR30C 

3907 

3945 

3496 

3156 

3728 

3377 

3085 

3703 

3495 

3240 

W6R30 

3310 

3412 

2935 

2587 

3263 

2868 

2534 

3348 

3092 

2795 

Wfift30F 

2804 

2912 

2431 

2083 

2789 

2380 

2032 

3096 

2786 

2445 

WLC 

317 

313 

328 

480 

826 

765 

847 

1803 

1695 

1564 

"Wi ~ 

312 

282 

229 

323 

775 

658 

682 

1866 

1732 

1569 

WLF 

408 

302 

157 

171 

703 

518 

470 

1970 

1776 

1561 

WFL15C 

160 

764 

731 

853 

1413 

1292 

1318 

1334 

1270 

1191 

WFL15 

102 

620 

561 

653 

1235 

1096 

1092 

1398 

1302 

1187 

WFL15F 

108 

534 

445 

503 

1105 

946 

910 

1430 

1314 

1177 

WFL30C 

115 

1067 

1026 

1128 

1738 

1607 

1599 

939 

893 

840 

WFL30 

40 

996 

921 

989 

1655 

1496 

1450 

960 

890 

811 

WFL30F 

16 

918 

829 

877 

1563 

1394 

1326 

1042 

954 

849 

WBL15C 

>87 

33 

138 

336 

314 

365 

543 

2573 

2413 

2218 

WBL15 

736 

66 

17 

97 

291 

214 

272 

2498 

2270 

2019 

WBL15F 

792 

234 

45 

15 

403 

210 

154 

2506 

2212 

1909 

W6L30C 

1428 

116 

189 

323 

1> 

100 

298 

3256 

2982 

2687 

WBL30 

1408 

200 

117 

143 

65 

10 

80 

3130 

2788 

2449 

WBL30F 

1399 

359 

152 

82 

222 

51 

7 

3089 

2681 

2292 

WFC 

1080 

2650 

2517 

2485 

3285 

3056 

2888 

16 

40 

133 

"WF- 

1034 

2576 

2383 

2297 

3111 

2838 

2626 

&8 

10 

27 

WFF 

1001 

2461 

2216 

2084 

2906 

2597 

2349 

225 

59 

TF- 

1533 

3047 

2966 

3002 

3758 

3585 

3481 

399 

637 

840 

CFR 

2651 

4179 

3950 

3832 

4630 

4343 

4117 

1073 

1173 

1260 

CFL 

788 

1630 

1613 

1747 

2277 

2170 

2208 

628 

686 

787 

"CR- 

4349 

4949 

4500 

4148 

4704 

4359 

4075 

2881 

2751 

2626 

~CE 

805 

711 

968 

1274 

1038 

1221 

1521 

2009 

1939 

1856 

CBR 

4820 

3768 

3617 

3457 

3123 

3102 

3136 

4474 

4270 

4041 

CBL 

2123 

1007 

1270 

1520 

920 

1215 

1619 

3917 

3771 

3572 

"CB- 

3972 

2298 

2219 

2177 

1625 

1688 

1864 

5382 

5026 

4615 

"DF- 

2397 

3211 

3196 

3334 

3828 

3737 

3795 

1585 

1885 

2096 

“BB 

3851 

2785 

3052 

3306 

2704 

3003 

3411 

4115 

4299 

4352 

107 


CF 

cc 

Li_ 

o 

"CFU 

"CR 

"cr — 

CBR 

CBL 

"CB 

"OF 

"OB 

Nulllemp 

3713 

— 372T 

3323 

3427 

3172 

3380 

3104 

3062 

5036 

5135 

TC 

2041 

2103 

1745 

2917 

2434 

4190 

3882 

4164 

3358 

5489 

~r~ 

2692 

2738 

2316 

2956 

2541 

3569 

3215 

3349 

4015 

5102 

FRC 

2139 

2081 

2629 

1837 

3902 

2634 

4100 

3750 

3396 

4313 

TR 

2764 

2748 

2770 

2464 

3377 

2803 

3387 

3233 

4063 

4518 

FLC 

1905 

2581 

1511 

4287 

1460 

4708 

2772 

4350 

3224 

4707 

T r~ 

2633 

2907 

2243 

3569 

2122 

3684 

2688 

3366 

3956 

4673 

TC 

2959 

3795 

1971 

4107 

1760 

3820 

1828 

2772 

3684 

3825 

L 

3348 

3662 

2732 

3508 

2577 

3331 

2503 

2735 

4429 

4484 

RC- 

2952 

2048 

4136 

1622 

4535 

1639 

4051 

2685 

3365 

3412 

"R- 

3093 

2721 

3323 

2395 

3372 

2388 

3160 

2582 

4036 

4143 

"EC 

4522 

4276 

4100 

2930 

3093 

1993 

1751 

1693 

5509 

3598 

"B 

3742 

3680 

3348 

3150 

3083 

2999 

2735 

2685 

4981 

4730 

BRC 

4430 

3296 

4254 

2098 

4003 

2067 

2617 

1779 

4571 

3796 

BR 

3742 

3380 

3454 

2832 

3271 

2761 

2787 

2469 

4683 

4504 

BLC 

4310 

4450 

3016 

4122 

2097 

3077 

1961 

1963 

4729 

3992 

"BH 

3768 

3822 

3066 

3478 

2803 

3223 

2691 

2649 

4779 

4724 

WRC 

2719 

1349 

4099 

513 

5178 

1166 

4270 

2616 

2638 

2835 

WR 

2582 

1402 

3916 

748 

4861 

1263 

4167 

2629 

2701 

2954 

WRF 

2640 

1592 

3828 

998 

4597 

1369 

3979 

2557 

2899 

3096 

WFR15C 

1949 

897 

3307 

601 

4624 

1676 

4554 

3154 

2126 

3191 

WFR15 

1971 

1077 

3251 

761 

4494 

1654 

4374 

3086 

2334 

3237 

WFR15F 

2074 

1332 

3246 

1024 

4291 

1695 

4147 

2975 

2595 

3312 

WFR30C 

1477 

715 

2793 

787 

4250 

2124 

4756 

3624 

1946 

3577 

WFR30 

1519 

893 

2763 

901 

4164 

2118 

4614 

3578 

2130 

3613 

WFR30F 

1639 

1115 

2793 

1025 

4056 

2054 

4394 

3418 

2376 

3625 

WBR15C 

3042 

1622 

4390 

530 

5143 

893 

3827 

2207 

2843 

2478 

WBR15 

2912 

1608 

4172 

720 

4827 

1043 

3753 

2209 

2843 

2660 

WBR15F 

2772 

1676 

3818 

980 

4321 

1225 

3503 

2159 

2921 

2860 

WBR30C 

3585 

2061 

4835 

667 

5130 

670 

3392 

1772 

3272 

2257 

WBR30 

3382 

2012 

4416 

844 

4675 

923 

3229 

1757 

3235 

2538 

WBR30F 

3312 

2102 

4080 

1104 

4207 

1155 

2983 

1699 

3333 

2784 

WLC 

2217 

3335 

925 

4817 

556 

4446 

1514 

3168 

2566 

3325 

"W r~ 

2300 

3376 

1094 

4626 

795 

4279 

1533 

3071 

2737 

3372 

WLF 

2448 

3416 

1326 

4330 

1095 

3999 

1589 

2867 

2969 

3444 

WFL15C 

1746 

2882 

572 

4674 

599 

4917 

2041 

3803 

2195 

3782 

WFL15 

1842 

2934 

764 

4560 

695 

4657 

1901 

3577 

2405 

3708 

WFL15F 

1894 

2940 

944 

4364 

849 

4357 

1821 

3395 

2571 

3612 

WFL30C 

1387 

2489 

447 

4377 

662 

5000 

2342 

4108 

2064 

4027 

WFL30 

1434 

2494 

608 

4274 

769 

4805 

2287 

3983 

2229 

4012 

WFL30F 

1540 

2578 

810 

4236 

885 

4659 

2213 

3871 

2449 

4002 

WBL15C 

3027 

4121 

1591 

5141 

624 

4048 

1040 

2572 

3304 

2995 

WBL15 

3000 

3970 

1626 

4666 

921 

3823 

1199 

2421 

3339 

3174 

WBL15F 

3062 

3904 

1784 

4320 

1305 

3709 

1477 

2403 

3497 

3470 

WBL30C 

3762 

4636 

2300 

4824 

939 

3309 

759 

1795 

3983 

2740 

WBL30 

3686 

4438 

2262 

4448 

1245 

3237 

1099 

1803 

3943 

3050 

WBL30F 

3739 

4351 

2399 

4247 

1590 

3256 

1444 

1912 

4112 

3457 

WFC 

530 

1092 

746 

2850 

2117 

4367 

3921 

5181 

1771 

4388 

"WF- 

778 

1230 

848 

2818 

2087 

4291 

3865 

4953 

2075 

4626 

WFF 

1057 

1405 

987 

2799 

2084 

4188 

3752 

4678 

2368 

4775 

CF- 

21 

721 

995 

2623 

2482 

4126 

4264 

5248 

944 

3583 

CFR 

835 

19 

2209 

1191 

3686 

2766 

4760 

4286 

946 

3265 

CFL 

968 

2106 

14 

4022 

1049 

5063 

2891 

4669 

1413 

3838 

”CR 

2765 

1237 

4-1 09 

13 

5280 

1152 

4288 

2690 

2498 

2719 

"Cl^ 

2437 

3585 

987 

5269 

20 

4698 

1408 

3244 

2678 

3265 

CBR 

4296 

2856 

5166 

1096 

4515 

11 

2497 

941 

3643 

1204 

CBL 

4291 

4723 

2927 

4491 

1208 

2716 

22 

1276 

3874 

1557 

"CB- 

5450 

4460 

4808 

2774 

3201 

1009 

1053 

5 

4229 

1352 

"DF- 

833 

769 

1297 

2337 

2598 

3468 

3718 

4002 

18 

2193 

”BB 

3331 

2915 

3637 

2625 

2908 

1238 

1446 

1440 

2062 

""S3 

APPENDIX  B 

TABLE  OF  MODIFIED  EUCLIDEAN  DISTANCE  OF  EACH  SCENARIO 
RESPECT  TO  THE  DATABASE  OF  SCENARIO  TEMPLATES  WITH  ONE 

SENSOR  MALFUNCTIONING 


||N  ullTemp 

TC 

T 

"FRC 

TR- 

FLC 

TE 

TC 

T~ 

RC- 

Null  lemp 

2 

1270 

245 

1021 

255 

1453 

227 

711 

155 

1295 

TC- 

i 35 

489 

250 

786 

570 

1752 

854 

1414 

880 

1996 

F 

209 

687 

66 

790 

284 

1444 

386 

918 

362 

1502 

FRC 

991 

1113 

728 

28 

260 

2318 

1198 

1730 

1152 

1248 

T R- 

306 

1040 

323 

239 

5 

1759 

541 

1045 

467 

1059 

FLC 

782 

1540 

785 

1661 

1009 

637 

413 

681 

699 

2023 

TL 

130 

1274 

315 

1155 

399 

967 

123 

539 

203 

1423 

TC 

678 

1916 

921 

1697 

931 

1147 

545 

7 

219 

1971 

T 

102 

1364 

345 

1121 

355 

1317 

247 

307 

“5 

1395 

RC- 

1243 

2481 

1486 

1058 

962 

2750 

1490 

1982 

1404 

"R 

207 

1465 

450 

742 

250 

1714 

454 

946 

368 

4>0 

"BC- 

806 

2110 

1063 

1867 

1081 

2313 

1053 

1545 

967 

2099 

"B- 

36 

1306 

279 

1055 

289 

1543 

283 

775 

197 

1329 

BRC 

929 

2233 

1186 

1950 

1192 

2436 

1176 

1668 

1090 

1282 

"BR 

101 

1405 

358 

1142 

370 

1608 

348 

840 

262 

1038 

BLC 

774 

2044 

1017 

1793 

1027 

2237 

1021 

765 

583 

2067 

BL 

68 

1372 

325 

1129 

343 

1559 

315 

535 

101 

1361 

WRC 

2159 

2691 

2096 

1056 

1414 

3538 

2368 

2868 

2312 

478 

"WR- 

1827 

2557 

1854 

918 

1176 

3298 

2066 

2566 

1988 

254 

WRF 

1417 

2299 

1510 

774 

894 

2906 

1660 

2156 

1578 

126 

WFR15C 

2235 

2309 

1948 

836 

1364 

3544 

2438 

2974 

2396 

598 

WFR15 

1880 

2194 

1707 

687 

1105 

3187 

2073 

2589 

2033 

389 

WFR15F 

1483 

2007 

1410 

494 

794 

2900 

1710 

2222 

1644 

280 

WFR30C 

2173 

1891 

1704 

692 

1290 

3300 

2326 

2882 

2326 

796 

WFR30 

1862 

1752 

1479 

479 

1013 

3043 

2027 

2571 

2015 

689 

WFR30F 

1518 

1618 

1235 

339 

755 

2809 

1717 

2257 

1679 

557 

WBR15C 

2288 

3200 

2395 

1559 

1727 

3777 

2531 

3027 

2449 

563 

WBR15 

1835 

2899 

2008 

1352 

1396 

3342 

2082 

2574 

1996 

302 

WBR15F 

1330 

2468 

1531 

1123 

1025 

2837 

1577 

2069 

1491 

163 

WBR30C 

2308 

3478 

2523 

2039 

1983 

3815 

2555 

3047 

2469 

763 

WBR30 

1714 

2926 

1943 

1723 

1525 

3221 

1961 

2453 

1875 

529 

WBR30F 

1147 

2399 

1390 

1410 

1076 

2654 

1394 

1886 

1308 

380 

WLC 

1576 

2562 

1699 

2535 

1819 

1321 

1181 

269 

885 

2869 

"WH 

1324 

2406 

1495 

2307 

1571 

1221 

981 

145 

681 

2617 

W 

966 

2104 

1161 

1961 

1215 

1167 

737 

39 

411 

2259 

WFL15C 

1543 

2127 

1460 

2364 

1748 

1150 

1116 

338 

900 

2836 

WFL15 

1332 

2134 

1359 

2243 

1567 

1017 

921 

229 

737 

2625 

WFL15F 

1083 

2027 

1182 

2030 

1324 

940 

728 

142 

552 

2376 

WFL30C 

1500 

1750 

1255 

2091 

1605 

1029 

1053 

499 

953 

2761 

WFL30 

1336 

1746 

1167 

2015 

1483 

907 

897 

CO 

837 

2545 

WFL30F 

1101 

1733 

1042 

1934 

1308 

806 

700 

334 

658 

2394 

WBL15C 

1689 

2865 

1908 

2696 

1940 

1672 

1376 

438 

1006 

2982 

WBL15 

1256 

2488 

1499 

2275 

1509 

1539 

1057 

189 

629 

2549 

WBL15F 

912 

2188 

1169 

1973 

1187 

1479 

823 

63 

357 

2205 

WBL30C 

1680 

2916 

1923 

2699 

1933 

2073 

1525 

639 

1053 

2973 

WBL30 

1213 

2497 

1470 

2274 

1488 

2000 

1212 

416 

658 

2506 

WBL30F 

813 

2105 

1070 

1874 

1088 

1844 

904 

264 

358 

2106 

WFC 

1763 

1261 

1198 

786 

1070 

1626 

1434 

1782 

1712 

2182 

"WF- 

1443 

989 

892 

680 

856 

1546 

1206 

1642 

1460 

2062 

WFF 

1146 

786 

633 

589 

661 

1491 

1001 

1465 

1195 

1951 

CF- 

2806 

2288 

2241 

1337 

1933 

2297 

2345 

2277 

2485 

2117 

CFR 

2885 

2387 

2320 

1356 

1994 

3044 

2670 

3114 

2910 

1280 

CFL 

2494 

2024 

1943 

1815 

1943 

1937 

2031 

1255 

1835 

3287 

UR- 

3270 

2942 

2777 

1797 

2391 

4379 

3419 

3979 

3423 

1603 

"Cl- 

2572 

2776 

2303 

3131 

2669 

2063 

2121 

1231 

1865 

3743 

CBR 

3281 

4159 

3374 

2530 

2704 

4754 

3524 

3748 

3314 

1684 

CBL 

2975 

4233 

3232 

4036 

3250 

3034 

2688 

1788 

2316 

3926 

"CB- 

3158 

4458 

3415 

3935 

3337 

4479 

3363 

2717 

2731 

2845 

T3F- 

4125 

3607 

3560 

2596 

3234 

3566 

3662 

2852 

3450 

2482 

4885 

5645 

4950 

4030 

4258 

4814 

4550 

3638 

4210 

3236 

109 
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H IBB 

BRB 

BR  | BlC  | Bl  |WfiC  | WR 

"WRF — 

Null  lemp 

T5B" 

73B" 

38 

932 

W 

— 7m 

7 T 

— 2W 

— BBBB' 

1487 

"FB- 

911 

1469 

771 

1665 

822 

1483 

804 

2414 

2267 

1894 

F 

397 

943 

245 

1139 

296 

957 

278 

2294 

2037 

1568 

FRC 

831 

1703 

1023 

1907 

1064 

1707 

1056 

1336 

1209 

986 

TR- 

316 

1040 

342 

1230 

387 

1054 

375 

1595 

1358 

987 

FLC 

900 

1516 

818 

1710 

867 

1522 

847 

3063 

2742 

2217 

FL 

298 

864 

166 

1060 

217 

878 

199 

2541 

2184 

1633 

TB 

836 

1412 

714 

1608 

765 

710 

479 

3073 

2712 

2163 

T~ 

284 

836 

138 

1032 

189 

558 

61 

2497 

2136 

1587 

RB 

469 

1977 

1279 

1287 

1014 

1991 

1312 

564 

311 

110 

"R- 

5 

941 

243 

807 

174 

955 

276 

1368 

1047 

634 

"BB- 

996 

4 

502 

866 

629 

1130 

831 

2667 

2450 

2081 

1 

226 

470 

4 

744 

57 

740 

101 

2285 

1964 

1461 

BRC 

727 

949 

797 

5 

452 

1677 

998 

1518 

1405 

1200 

BR 

143 

631 

89 

425 

2 

849 

170 

1892 

1603 

1162 

BLC 

964 

1134 

742 

1704 

861 

10 

423 

3169 

2808 

2259 

"BE 

258 

758 

96 

998 

155 

400 

1 

2497 

2128 

1571 

WRC 

1217 

2485 

2099 

1433 

1700 

2907 

2228 

14 

27 

132 

WR 

929 

2323 

1807 

1399 

1456 

2575 

1896 

86 

^7 

32 

WRF 

627 

2015 

1421 

1251 

1128 

2165 

1486 

224 

87 

“ 5 

WFR15C 

1317 

2799 

2231 

1891 

1888 

2983 

2304 

166 

177 

262 

WFR15 

1018 

2512 

1892 

1752 

1599 

2628 

1949 

189 

134 

143 

WFR15F 

737 

2183 

1511 

1619 

1286 

2231 

1552 

296 

179 

104 

WFR30C 

1359 

2873 

2201 

2261 

1958 

2921 

2242 

498 

487 

508 

WFR30 

1132 

2562 

1890 

2128 

1705 

2610 

1931 

501 

454 

425 

WFR30F 

872 

2252 

1554 

1918 

1405 

2266 

1587 

529 

436 

347 

WBR15C 

1326 

2306 

2156 

1134 

1703 

3032 

2357 

115 

130 

249 

WBR15 

945 

2059 

1751 

985 

1342 

2583 

1904 

162 

97 

116 

WBR15F 

578 

1690 

1278 

786 

933 

2078 

1399 

293 

162 

79 

WBR30C 

1406 

1904 

2080 

918 

1651 

3024 

2377 

429 

424 

519 

WBR30 

944 

1590 

1550 

602 

1147 

2446 

1783 

439 

378 

373 

WBR30F 

535 

1301 

1047 

397 

696 

1891 

1216 

562 

435 

318 

WLC 

1712 

2156 

1584 

2506 

1663 

988 

1205 

3941 

3600 

3061 

WL 

1464 

1992 

1348 

2254 

1411 

952 

1001 

3701 

3352 

2809 

WLF 

1116 

1656 

994 

1896 

1053 

758 

695 

3349 

2996 

2451 

WFL15C 

1679 

2233 

1571 

2473 

1630 

1247 

1240 

3826 

3525 

3010 

WFL15 

1468 

2044 

1364 

2262 

1419 

1200 

1081 

3673 

3348 

2817 

WFL15F 

1223 

1817 

1119 

2013 

1170 

1115 

884 

3442 

3105 

2568 

WFL30C 

1626 

2234 

1536 

2430 

1587 

1532 

1301 

3605 

3350 

2875 

WFL30 

1442 

2070 

1372 

2264 

1421 

1500 

1185 

3467 

3188 

2697 

WFL30F 

1239 

1835 

1137 

2031 

1188 

1397 

998 

3390 

3085 

2568 

WBL15C 

1831 

2013 

1649 

2615 

1774 

733 

1262 

4076 

3719 

3174 

WBL15 

1408 

1712 

1240 

2182 

1341 

536 

869 

3649 

3288 

2741 

WBL15F 

1074 

1514 

924 

1842 

999 

444 

577 

3341 

2972 

2415 

WBL30C 

1836 

1624 

1560 

2502 

1731 

512 

1229 

4007 

3662 

3141 

WBL30 

1383 

1373 

1137 

2085 

1280 

257 

798 

3606 

3245 

2704 

WBL30F 

991 

1177 

777 

1693 

884 

153 

458 

3210 

2849 

2304 

WFC 

1613 

2497 

1799 

2639 

1826 

2503 

1828 

2196 

2123 

1914 

"WF- 

1371 

2177 

1479 

2323 

1510 

2191 

1512 

2156 

2069 

1826 

WFF 

1148 

1880 

1182 

2072 

1229 

1894 

1215 

2143 

2034 

1751 

"BF- 

2224 

3540 

2842 

3558 

2805 

3414 

2817 

2025 

1966 

1863 

CFR 

1943 

3449 

2881 

2541 

2538 

3633 

2954 

762 

823 

932 

CFL 

2470 

3184 

2522 

3422 

2579 

2154 

2175 

3413 

3320 

3055 

"BR 

2332 

2794 

3026 

1916 

2623 

3978 

3339 

483 

704 

1007 

"BL 

2662 

2372 

2420 

3332 

2599 

1388 

2121 

4489 

4258 

3833 

CBR 

2371 

1969 

2869 

1927 

2626 

2953 

3142 

1052 

1107 

1294 

CBL 

2975 

1609 

2555 

2417 

2606 

1759 

2536 

4222 

4081 

3830 

"BB 

2696 

1774 

2734 

1796 

2521 

1890 

2707 

2827 

2768 

2711 

"OF 

3147 

4577 

4097 

3653 

3740 

3741 

3790 

1894 

1975 

2108 

"BB- 

3905 

3485 

4457 

3529 

4228 

3669 

4446 

2534 

2605 

2812 
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WFR15F 

WFR30C 
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WFR30F 

WBR15C 

WBR15 

WBR15F 
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Null  lemp 

2345 

2056 

1577 

2397 

2063 

1651 

2481 

1935 

1359 

2353 

TC- 

1898 

1799 

1546 

1622 

1406 

1228 

2948 

2472 

1968 

2990 

F 

1994 

1805 

1440 

1864 

1592 

1304 

2590 

2072 

1524 

2532 

FRC 

972 

819 

626 

848 

604 

418 

1944 

1614 

1324 

2228 

TFT- 

1407 

1180 

845 

1397 

1101 

793 

1965 

1541 

1123 

2073 

FLC 

2851 

2622 

2207 

2713 

2471 

2143 

3207 

2663 

2093 

3081 

”FE 

2449 

2174 

1707 

2441 

2137 

1745 

2623 

2073 

1493 

2485 

TC 

3021 

2732 

2253 

3043 

2739 

2327 

3157 

2611 

2035 

3029 

L 

2445 

2156 

1677 

2491 

2163 

1751 

2581 

2035 

1459 

2453 

RC- 

626 

451 

288 

832 

736 

570 

674 

350 

166 

756 

tt- 

1358 

1111 

756 

1468 

1224 

910 

1462 

1002 

582 

1416 

~BC~ 

2963 

2784 

2337 

3171 

2871 

2493 

2281 

2031 

1677 

1901 

~s~ 

2319 

2060 

1591 

2413 

2087 

1685 

2261 

1787 

1267 

2087 

BRC 

1942 

1907 

1648 

2342 

2270 

2036 

1160 

954 

728 

942 

BR 

1992 

1783 

1378 

2160 

1916 

1564 

1832 

1388 

922 

1672 

BLC 

3117 

2828 

2349 

3171 

2835 

2423 

3253 

2707 

2131 

3091 

"BE 

2453 

2160 

1675 

2509 

2171 

1755 

2561 

2011 

1431 

2419 

WRC 

120 

137 

184 

382 

420 

420 

240 

176 

254 

476 

156 

107 

94 

398 

390 

338 

330 

180 

176 

546 

WRF 

268 

165 

82 

482 

424 

316 

454 

228 

136 

628 

WFR15C 

4 

25 

112 

90 

128 

176 

648 

514 

534 

962 

WFR15 

49 

10 

29 

141 

129 

117 

631 

443 

405 

913 

WFR15F 

164 

63 

4 

256 

178 

96 

720 

474 

364 

976 

WFR30C 

116 

123 

198 

20 

22 

84 

1122 

914 

854 

1452 

WFR30 

151 

112 

127 

67 

13 

19 

1095 

865 

771 

1413 

WFR30F 

215 

132 

87 

155 

57 

V 

1083 

815 

671 

1371 

WBR15C 

467 

498 

513 

867 

913 

887 

25 

35 

171 

141 

WBR15 

460 

431 

384 

824 

828 

752 

98 

10 

48 

190 

WBR15F 

525 

446 

331 

837 

793 

665 

255 

75 

307 

WBR30C 

939 

970 

933 

1405 

1443 

1383 

65 

113 

251 

3 

WBR30 

873 

864 

769 

1293 

1295 

1191 

139 

81 

105 

WBR30F 

900 

839 

676 

1258 

1210 

1052 

344 

172 

70 

270 

WLC 

3809 

3550 

3101 

3739 

3477 

3105 

4055 

3509 

2933 

3913 

~wl 

3601 

3330 

2869 

3563 

3289 

2901 

3803 

3257 

2681 

3669 

WLF 

3265 

2988 

2521 

3251 

2963 

2567 

3445 

2899 

2323 

3313 

WFL15C 

3558 

3351 

2956 

3374 

3146 

2842 

4008 

3466 

2894 

3886 

WFL15 

3477 

3242 

2817 

3351 

3105 

2765 

3811 

3265 

2689 

3681 

WFL15F 

3294 

3041 

2598 

3214 

2952 

2588 

3562 

3016 

2440 

3434 

WFL30C 

3233 

3068 

2725 

2973 

2771 

2521 

3881 

3357 

2805 

3803 

WFL30 

3143 

2958 

2595 

2915 

2703 

2433 

3691 

3161 

2607 

3595 

WFL30F 

3138 

2925 

2524 

2970 

2736 

2424 

3566 

3024 

2452 

3448 

WBL15C 

4010 

3727 

3254 

4002 

3718 

3314 

4160 

3618 

3044 

3990 

WBL15 

3599 

3310 

2831 

3615 

3317 

2905 

3727 

3185 

2611 

3569 

WBL15F 

3297 

3004 

2519 

3325 

3015 

2599 

3405 

2855 

2275 

3255 

WBL30C 

3999 

3722 

3247 

4035 

3737 

3329 

4019 

3521 

2977 

3789 

WBL30 

3586 

3299 

2816 

3630 

3314 

2900 

3628 

3106 

2544 

3418 

WBL30F 

3186 

2899 

2416 

3238 

2914 
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3244 

2714 

2148 

3062 

WFC 

1638 

1567 

1440 

1308 

1164 

1070 

2882 

2538 

2222 

3144 

~WF 

1606 

1531 

1378 

1286 

1128 

1016 

2810 

2440 

2084 

3028 

WFF 

1611 

1520 

1331 

1307 

1123 

983 

2761 

2363 

1967 

2937 

TTF- 

1479 

1420 

1393 

1155 

1077 

1073 

2719 

2437 

2265 

3047 

CFR 

416 

507 

662 

312 

430 

568 

1326 

1190 

1206 

1640 

CFL 

2857 

2776 

2603 

2511 

2357 

2229 

4065 

3681 

3299 

4265 

CR 

543 

754 

951 

697 

839 

1007 

603 

757 

1005 

755 

~CT~ 

4159 

4018 

3697 

3909 

3727 

3499 

4673 

4219 

3719 

4463 

CBR 

1504 

1557 

1568 

1926 
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1934 

776 

920 

1132 

680 

CBL 

4658 

4563 

4198 

4932 

4778 

4464 

3710 

3568 

3324 

3260 
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3387 

3400 

3219 

3849 

3839 
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2263 

2231 

2179 

1853 

DF 

1586 

1695 

1862 

1486 

1638 
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2420 

2312 
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2970 

3031 

3058 

3338 
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3414 
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2452 

2678 
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2497 
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1822 
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1881 

2068 

1973 

F 

1993 

1409 

2098 

1777 

1338 

2117 

1873 

1495 

1994 

1827 
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1991 

1645 

2934 

2607 

2158 

2979 

2745 

2345 

2834 

2681 

FR 

1688 

1230 

2305 

1944 

1477 

2496 

2188 

1732 

2465 

2252 

FLC 

2520 

1946 

1223 

994 

801 

1296 

1022 

778 

1225 

1032 

TT~ 

1914 

1330 

1513 

1196 

839 

1728 

1396 

1006 

1745 

1508 

TC- 

2462 

1878 

629 

380 

157 

992 

728 

434 

1191 

1028 

L 

1886 

1302 

1385 

1052 

655 

1694 

1376 

956 

1803 

1586 

RC 

575 

445 

3260 

2887 

2414 

3533 

3179 

2699 

3582 

3329 

~R 

1003 

605 

2224 

1851 

1378 

2497 

2143 

1663 

2546 

2293 

"BC- 

1574 

1252 

2633 

2326 

1897 

3054 

2720 

2240 

3129 

2896 
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1570 

1048 

2033 

1666 

1197 

2322 

1970 

1490 

2373 
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8RC 

597 

331 

2946 

2573 

2100 

3221 

2865 

2385 

3274 

3019 

“BR- 

1175 

693 

2118 

1745 

1272 

2393 

2037 

1557 

2446 

2191 

BLC 

2558 

1974 

1453 

1270 

997 

2028 

1840 

1492 

2331 

2244 

"BU- 

1852 

1268 

1657 

1328 

911 

2000 

1696 

1264 

2137 

1934 

WRC 

507 

571 

4096 

3743 

3288 

4259 

3967 

3525 

4200 

4001 

WR 

517 

515 

3832 

3467 

2998 

4041 

3727 

3263 

4018 

3801 

WRF 

525 

445 

3428 

3059 

2588 

3663 

3335 

2863 

3664 

3435 

WFR15C 

969 

963 

4186 

3857 

3406 

4201 

3973 

3581 

4042 

3893 

WFR15 

874 

828 

3793 

3456 

3009 

3874 

3616 

3206 

3763 

3590 

WFR15F 

879 

769 

3470 

3117 

2654 

3615 

3329 

2889 

3548 

3353 

WFR30C 

1405 

1327 

4026 

3729 

3302 

3899 

3729 

3399 

3652 

3547 

WFR30 

1340 

1236 

3733 

3424 

2991 

3668 

3472 

3118 

3461 

3336 

WFR30F 

1254 

1104 

3463 

3138 

2689 

3462 

3238 

2854 

3299 

3152 

WBR15C 

196 

344 

4297 

3928 

3457 

4536 

4206 

3734 

4541 

4310 

WBR15 

163 

223 

3852 

3479 

3006 

4115 

3771 

3291 

4144 

3901 

WBR15F 

188 

142 

3347 

2974 

2501 

3614 

3266 

2786 

3651 

3404 

WBR30C 

60 

244 

4309 

3936 

3463 

4594 

4244 

3764 

4635 

4386 

WBR30 

8 

66 

3723 

3350 

2877 

4002 

3650 

3170 

4047 

3796 

WBR30F 

91 

11 

3162 

2789 

2316 

3437 

3083 

2603 

3486 

3233 

WLC 

3360 

2776 

333 

226 

205 

720 

564 

430 

963 

898 

"WT- 

3108 

2524 

361 

206 

133 

736 

544 

370 

975 

878 

WLF 

2750 

2166 

453 

256 

109 

820 

600 

370 

1039 

916 

WFL15C 

3327 

2743 

472 

355 

312 

629 

511 

423 

732 

689 

WFL15 

3116 

2532 

449 

298 

225 

660 

488 

350 

783 

694 

WFL15F 

2867 

2283 

494 

305 

134 

737 

521 

329 

864 

737 

WFL30C 

3262 

2686 

727 

594 

517 

722 

610 

526 

689 

642 

WFL30 

3052 

2486 

725 

572 

473 

746 

596 

478 

715 

634 

WFL30F 

2885 

2301 

720 

535 

400 

805 

603 

429 

796 

671 

WBL15C 

3463 

2883 

446 

375 

362 

1021 

859 

681 

1342 

1279 

WBL15 

3030 

2450 

433 

298 

203 

962 

774 

552 

1273 

1186 

WBL15F 

2696 

2112 

541 

348 

171 

1008 

796 

532 

1293 

1182 

WBL30C 

3316 

2776 

741 

674 

613 

1406 

1248 

1020 

1751 

1694 

WBL30 

2913 

2355 

790 

661 

504 

1397 

1221 

945 

1726 

1651 

WBL30F 

2533 

1967 

862 

673 

440 

1393 

1193 

881 

1690 

1591 

WFC 

2871 

2485 

2356 

2159 

1950 

2149 

2011 

1839 

1854 

1773 

"WF- 

2709 

2279 

2320 

2107 

1858 

2115 

1975 

1779 

1824 

1741 

WFF 

2574 

2106 

2253 

2020 

1729 

2070 

1920 

1692 

1795 

1704 

CF 

2920 

2746 

2689 

2520 

2373 

2494 

2368 

2238 

2219 

2150 

CFR 

1641 

1627 

3818 

3601 

3342 

3611 

3473 

3271 

3316 

3235 

CFL 

3924 

3462 

1319 

1242 

1223 

1266 

1262 

1252 

1199 

1252 

UR- 

942 

1170 

5097 

4804 

4379 

4970 

4808 

4486 

4711 

4612 

~cr~ 

4050 

3548 

951 

1000 

1077 

1236 

1238 

1222 

1337 

1402 

CBR 

915 

1175 

4570 

4353 

4010 

5095 

4855 

4431 

5216 

5075 

CBL 

3043 

2819 

1706 

1665 

1662 

2407 

2243 

2045 

2748 

2685 

"CB 

1784 

1764 

3181 

3062 

2849 

3894 

3738 

3414 

4247 

4192 

~ur~ 

2743 

2753 

2900 

2827 

2814 

2851 

2855 

2851 

2790 

2851 

~0B 

2473 

2743 

3498 

3469 

3488 

4155 

4023 

3845 

4436 

4403 

113 


WFL30F 

WBL15C  | W6L15 

WBL15F 

WBL30C 

WBL30 

"WBE30T 

WFC 

"WF 

“WFF 

Null  lemp 

1670 

— T73T 

— T23T 

887 

1689 

1280 

832 

2676 

2146 

1681 

"FC- 

1771 

2419 

1938 

1594 

2394 

1943 

1553 

1609 

1171 

828 

F 

1551 

1941 

1448 

1094 

1896 

1437 

1039 

1999 

1507 

1094 

FRC 

2415 

2737 

2244 

1888 

2660 

2205 

1809 

1473 

1157 

962 

TR- 

1928 

2086 

1581 

1217 

2021 

1554 

1148 

1884 

1460 

1139 

FLC 

784 

1254 

1061 

961 

1709 

1454 

1260 

1748 

1552 

1313 

FL 

1182 

1382 

1009 

755 

1537 

1158 

840 

2224 

1818 

1435 

tc- 

844 

520 

237 

121 

745 

464 

304 

2490 

2196 

1879 

T 

1292 

1180 

735 

449 

1183 

774 

450 

2564 

2098 

1671 

RC 

2965 

3023 

2518 

2154 

2958 

2491 

2085 

2987 

2729 

2434 

R 

1929 

1987 

1482 

1118 

1922 

1455 

1049 

2543 

2117 

1714 

"BC“ 

2530 

2068 

1767 

1491 

1615 

1426 

1276 

3572 

3028 

2551 

B 

1758 

1758 

1277 

921 

1631 

1204 

836 

2758 

2216 

1743 

BRC 

2653 

2709 

2204 

1840 

2560 

2135 

1771 

3627 

3139 

2666 

6ft 

1825 

1881 

1376 

1012 

1792 

1337 

943 

2839 

2317 

1842 

BLC 

2056 

876 

619 

479 

563 

344 

240 

3452 

2910 

2437 

"BE 

1632 

1380 

915 

599 

1275 

848 

510 

2818 

2274 

1797 

WftC 

3693 

3891 

3398 

3044 

3798 

3363 

2989 

2779 

2619 

2442 

WR 

3473 

3607 

3102 

2738 

3514 

3061 

2669 

2767 

2575 

2366 

WRF 

3095 

3197 

2692 

2328 

3116 

2657 

2259 

2647 

2417 

2174 

WFR15C 

3633 

4015 

3510 

3146 

3930 

3473 

3077 

2355 

2207 

2060 

WFR15 

3308 

3612 

3119 

2765 

3555 

3102 

2710 

2274 

2114 

1949 

WFR15F 

3047 

3263 

2758 

2394 

3194 

2729 

2325 

2229 

2021 

1822 

WFR30C 

3333 

3905 

3412 

3058 

3856 

3399 

3003 

1917 

1779 

1650 

WFR30 

3102 

3594 

3101 

2747 

3545 

3088 

2692 

1814 

1654 

1513 

WFR30F 

2894 

3298 

2793 

2429 

3233 

2766 

2360 

1818 

1614 

1439 

WBR15C 

3968 

4060 

3559 

3195 

3905 

3484 

3124 

3444 

3250 

3023 

WBR15 

3547 

3615 

3110 

2746 

3490 

3053 

2677 

3267 

3043 

2784 

WBR15F 

3046 

3110 

2605 

2241 

3001 

2556 

2172 

3012 

2742 

2445 

WBR30C 

4026 

4024 

3551 

3187 

3779 

3410 

3102 

3948 

3710 

3423 

WBR30 

3434 

3462 

2973 

2609 

3273 

2872 

2532 

3598 

3312 

2983 

WBR30F 

2869 

2919 

2418 

2054 

2780 

2351 

1983 

3249 

2913 

2548 

WLC 

838 

202 

153 

239 

545 

454 

506 

2602 

2450 

2231 

"Wi- 

786 

282 

157 

183 

637 

478 

462 

2546 

2366 

2127 

WLF 

784 

348 

145 

101 

621 

404 

320 

2482 

2240 

1959 

WFL15C 

655 

489 

374 

408 

898 

741 

725 

2147 

2013 

1814 

WFL15 

610 

462 

329 

343 

889 

714 

670 

2178 

2034 

1823 

WFL15F 

607 

493 

318 

292 

898 

689 

601 

2175 

2001 

1768 

WFL30C 

596 

814 

663 

657 

1259 

1066 

994 

1704 

1592 

1423 

WFL30 

548 

800 

643 

625 

1251 

1052 

962 

1718 

1604 

1425 

WFL30F 

533 

763 

588 

548 

1200 

987 

871 

1825 

1677 

1468 

WBL15C 

1197 

61 

102 

242 

212 

225 

365 

3049 

2853 

2602 

WBL15 

1080 

122 

29 

67 

247 

146 

180 

2906 

2648 

2355 

WBL15F 

1048 

296 

75 

15 

395 

184 

110 

2850 

2530 

2193 

WBL30C 

1592 

144 

193 

311 

25 

88 

266 

3424 

3142 

2833 

WBL30 

1525 

249 

144 

148 

76 

7 

59 

3339 

2971 

2602 

WBL30F 

1437 

421 

184 

88 

264 

71 

3 

3151 

2727 

2322 

WFC 

1629 

2427 

2186 

2046 

2802 

2515 

2289 

695 

649 

634 

~W~ 

1583 

2359 

2070 

1886 

2654 

2331 

2069 

771 

637 

554 

WFF 

1530 

2254 

1917 

1693 

2469 

2114 

1820 

892 

680 

533 

CF- 

2054 

2800 

2623 

2563 

3271 

3050 

2898 

1066 

1254 

1351 

CFR 

3077 

3849 

3548 

3354 

4104 

3783 

3523 

1657 

1731 

1738 

CFL 

1316 

1390 

1277 

1315 

1797 

1642 

1632 

1300 

1310 

1315 

"CR- 

4404 

4922 

4469 

4115 

4677 

4328 

4040 

2960 

2836 

2707 

"BE 

1442 

524 

709 

927 

565 

732 

1014 

2694 

2592 

2427 

CBR 

4799 

3813 

3648 

3476 

3182 

3147 

3165 

4393 

4191 

3966 

CBL 

2585 

1047 

1238 

1420 

800 

1055 

1419 

4459 

4265 

4000 

"CB- 

4048 

2338 

2271 

2237 

1649 

1726 

1914 

5512 

5148 

4731 

T5F- 

2925 

2971 

2860 

2902 

3348 

3209 

3219 

2257 

2509 

2624 

~BB~ 

4337 

2861 

3076 

3278 

2650 

2923 

3305 

4553 

4717 

4730 

114 


CF 

CFR 

"CFE 

"CR 

■cc — 

CBR 

"CBE 

"CB 

T3F 

dB 

Null  lemp 

3756 

3784 

3366 

3466 

3205 

3379 

3071 

3061 

5079 

5102 

TC 

2675 

2725 

2333 

2653 

3020 

3846 

3756 

3794 

3998 

5455 

F 

3079 

3107 

2703 

2911 

2928 

3488 

3278 

3268 

4402 

5183 

FRC 

2183 

2147 

2485 

1895 

3750 

2802 

4060 

3806 

3460 

4491 

TR- 

2792 

2780 

2754 

2460 

3407 

2863 

3423 

3281 

4097 

4592 

FLC 

2552 

3176 

2204 

3814 

2145 

4105 

2591 

3773 

3865 

4518 

FL 

3224 

3472 

2832 

3526 

2663 

3521 

2719 

3189 

4547 

4748 

TC 

3080 

3892 

2092 

4114 

1871 

3785 

1857 

2737 

3805 

3858 

T“ 

3444 

3756 

2812 

3556 

2627 

3371 

2517 

2733 

4525 

4542 

wr~ 

2965 

2061 

4133 

1635 

4502 

1648 

4018 

2652 

3378 

3421 

3157 

2817 

3425 

2439 

3466 

2388 

3198 

2608 

4134 

4161 

"BC- 

4624 

4364 

4188 

2984 

3149 

2051 

1765 

1703 

5599 

3666 

"B- 

3840 

3776 

3446 

3190 

3171 

2997 

2749 

2683 

5079 

4746 

"SRC 

4511 

3331 

4387 

2137 

4108 

2062 

2684 

1782 

4606 

3797 

BR 

3843 

3439 

3561 

2841 

3340 

2738 

2798 

2432 

4744 

4503 

BLC 

4408 

4546 

3114 

4162 

2185 

3075 

1975 

1961 

4827 

4008 

T5U- 

3858 

3912 

3154 

3558 

2843 

3235 

2689 

2647 

4869 

4740 

WRC 

2647 

1351 

4019 

543 

5102 

1190 

4226 

2640 

2622 

2829 

~WR 

2661 

1493 

3995 

819 

4934 

1292 

4208 

2658 

2796 

3003 

WRF 

2643 

1649 

3823 

1085 

4586 

1416 

3990 

2604 

2958 

3145 

WFR15C 

2237 

1097 

3617 

585 

4980 

1618 

4708 

3166 

2404 

3305 

WFR15 

2172 

1202 

3506 

778 

4703 

1597 

4451 

3061 

2491 

3250 

WFR15F 

2237 

1445 

3427 

1045 

4492 

1678 

4266 

3020 

2760 

3393 

WFR30C 

1833 

1011 

3151 

771 

4608 

2084 

4890 

3604 

2304 

3725 

WFR30 

1790 

1134 

3028 

880 

4415 

2041 

4651 

3521 

2427 

3686 

WFR30F 

1892 

1320 

3002 

1046 

4253 

2025 

4437 

3383 

2637 

3730 

WBR15C 

3312 

1802 

4670 

616 

5377 

917 

3957 

2231 

3079 

2616 

WBR15 

3197 

1843 

4455 

853 

5014 

1036 

3812 

2194 

3130 

2759 

WBR15F 

3086 

1926 

4130 

1092 

4533 

1197 

3545 

2123 

3221 

2936 

WBR30C 

3862 

2288 

5098 

786 

5299 

699 

3453 

1763 

3555 

2398 

WBR30 

3664 

2244 

4680 

966 

4805 

917 

3247 

1705 

3519 

2636 

WBR30F 

3501 

2295 

4259 

1263 

4322 

1232 

3090 

1764 

3580 

2971 

WEC- 

3050 

4158 

1692 

4790 

1205 

4241 

1441 

2867 

3405 

3420 

WL 

3022 

4074 

1756 

4626 

1417 

4185 

1577 

2903 

3469 

3560 

WLF 

3016 

3940 

1868 

4314 

1571 

3921 

1657 

2737 

3581 

3650 

WFLi'50 

2613 

3703 

1391 

4425 

1234 

4448 

1798 

3218 

3102 

3759 

WFL15 

2662 

3734 

1556 

4418 

1373 

4345 

1785 

3199 

3269 

3764 

WFL15F 

2693 

3703 

1705 

4291 

1544 

4190 

1830 

3142 

3418 

3813 

WFL30C 

2200 

3254 

1236 

4030 

1377 

4509 

2151 

3559 

2925 

4016 

WFL30 

2196 

3226 

1388 

3982 

1447 

4373 

2137 

3511 

3029 

3984 

WFL30F 

2371 

3367 

1613 

4049 

1574 

4302 

2100 

3448 

3324 

4063 

WBL15C 

3513 

4561 

2077 

5013 

1128 

3894 

1126 

2414 

3790 

3109 

WBL15 

3416 

4348 

2042 

4632 

1327 

3719 

1271 

2317 

3755 

3264 

WBL15F 

3414 

4224 

2134 

4366 

1607 

3667 

1549 

2347 

3849 

3572 

WBL30C 

3934 

4800 

2474 

4830 

1177 

3335 

863 

1827 

4155 

2838 

WBL30 

3915 

4635 

2511 

4509 

1420 

3234 

1152 

1774 

4210 

3169 

WBL30F 

3815 

4383 

2515 

4165 

1708 

3194 

1476 

1874 

4214 

3505 

WFC 

1273 

1745 

1445 

2363 

2814 

3784 

3746 

4592 

2522 

4317 

"WF 

1511 

1855 

1545 

2333 

2782 

3712 

3678 

4368 

2776 

4509 

WFF 

1724 

2026 

1654 

2362 

2751 

3659 

3591 

4149 

3035 

4660 

U_ 

0_ 

694 

1386 

1660 

2188 

3147 

3593 

4083 

4715 

1657 

3482 

CFR 

1409 

643 

2775 

819 

4256 

2296 

4542 

3816 

1576 

3129 

CFL 

1640 

2730 

T5S 

3542 

1721 

4487 

2699 

4093 

2085 

3694 

"CR- 

2838 

1326 

4182 

16 

5371 

1165 

4301 

2699 

2571 

2738 

"Cl- 

3068 

4108 

1686 

4692 

691 

3857 

1089 

2489 

3243 

2836 

CBR 

4221 

2775 

5135 

1017 

4666 

16 

2556 

998 

3610 

1245 

CBL 

4843 

5167 

3487 

4301 

1758 

2442 

68 

1002 

4386 

1599 

UR- 

5616 

4618 

4922 

2894 

3231 

1061 

1033 

0 

4381 

1398 

OF 

1505 

1393 

1969 

1857 

3270 

2892 

3526 

3426 

690 

2049 

T3B 

3769 

3327 

4081 

2439 

3466 

1090 

1548 

1334 

5500 

"H57 

APPENDIX  C 

TABLE  OF  MODIFIED  EUCLIDEAN  DISTANCE  OF  EACH  SCENARIO 
RESPECT  TO  THE  DATABASE  OF  SCENARIO  TEMPLATES  WITH  TWO 

SENSOR  MALFUNCTIONING 


NullTemp 

TC 

T 

TEC 

TR 

TCC 

TC 

TC 

T 

TC 

Null  lemp 

15 

1153 

196 

860 

188 

1430 

232 

724 

168 

1154 

TC- 

744 

494 

259 

687 

525 

1761 

863 

1423 

889 

1889 

F 

242 

668 

75 

703 

261 

1403 

393 

921 

387 

1419 

FRC 

927 

1175 

726 

14 

228 

2234 

1120 

1636 

1080 

1022 

TR- 

314 

1000 

307 

235 

11 

1693 

523 

1023 

467 

1067 

FlC 

806 

1480 

771 

1543 

961 

659 

437 

765 

759 

1983 

t r~ 

169 

1193 

302 

990 

338 

932 

136 

548 

234 

1346 

TC 

522 

1676 

727 

1379 

697 

1083 

413 

2^7 

299 

1699 

T~ 

85 

1261 

290 

942 

260 

1280 

218 

436 

94 

1224 

wr~ 

1219 

2291 

1386 

838 

842 

2634 

1436 

1928 

1372 

~R 

225 

1399 

430 

662 

220 

1658 

446 

934 

378 

456 

"BC- 

804 

1988 

1009 

1661 

979 

2237 

1025 

1513 

957 

1943 

T- 

49 

1233 

254 

906 

224 

1482 

270 

758 

202 

1226 

BRC 

903 

2087 

1108 

1720 

1066 

2336 

1124 

1612 

1056 

1140 

BR 

82 

1268 

287 

983 

281 

1515 

303 

791 

235 

1025 

BLC 

742 

1926 

947 

1599 

917 

2133 

963 

737 

559 

1881 

"BU- 

79 

1263 

284 

936 

254 

1496 

300 

516 

104 

1256 

WRC 

2212 

2618 

2087 

1043 

1441 

3555 

2413 

2921 

2365 

531 

WR 

1809 

2457 

1798 

866 

1140 

3206 

2022 

2518 

1962 

288 

WRF 

1456 

2210 

1487 

747 

907 

2853 

1669 

2165 

1609 

165 

WFR15C 

2162 

2190 

1851 

795 

1313 

3397 

2339 

2871 

2315 

595 

WFR15 

1840 

2072 

1629 

613 

1047 

3073 

2007 

2519 

1985 

439 

WFR15F 

1509 

1939 

1388 

496 

816 

2834 

1706 

2218 

1662 

306 

WFR30C 

2143 

1859 

1674 

662 

1260 

3214 

2274 

2822 

2288 

804 

WFR30 

1888 

1778 

1505 

505 

1039 

3069 

2053 

2597 

2041 

677 

WFR30F 

1498 

1596 

1215 

319 

735 

2733 

1675 

2207 

1651 

575 

WBR15C 

2279 

3075 

2334 

1454 

1672 

3694 

2496 

2988 
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3445 

3202 

WLC 

3106 

2548 

577 

478 

461 

922 

784 

666 

1137 

1086 

WL 

2844 

2286 

587 

442 

375 

950 

770 

604 

1175 

1088 

WLF 

2477 

1919 

684 

491 

346 

1029 

819 

597 

1232 

1117 

WFL15C 

3094 

2536 

699 

586 

549 

860 

750 

668 

969 

934 

WFL15 

2862 

2304 

699 

552 

1 481 

886 

726 

596 

989 

910 

WFL15F 

2589 

2031 

750 

571 

452 

955 

753 

571 

1046 

929 

WFL30C 

3011 

2453 

1000 

873 

792 

981 

883 

801 

930 

895 

WFL30 

2833 

2275 

998 

845 

738 

995 

857 

741 

944 

873 

WFL30F 

2687 

2129 

970 

797 

672 

1017 

841 

689 

986 

883 

WBL15C 

3178 

2624 

631 

560 

553 

1144 

996 

840 

1433 

1380 

WBL15 

2784 

2218 

655 

528 

437 

1140 

970 

764 

1423 

1350 

WBL15F' 

2377 

1819 

776 

583 

402 

1213 

1007 

745 

1468 

1361 

WBL30C” 

3033 

2527 

1070 

1009 

936 

1707 

1569 

1345 

2026 

1985 

WBL30 

2640 

2108 

1015 

894 

741 

1578 

1422 

1162 

1875 

1816 

WBL30F 

2365 

1815 

1038 

857 

636 

1545 

1357 

1059 

1822 

1733 

WFC 

2912 

2526 

2401 

2208 

2009 

2194 

2056 

1890 

1899 

1818 

"WF- 

2796 

2362 

2287 

2084 

1849 

2092 

1954 

1764 

1803 

1722 

WFF 

2566 

2098 

2189 

1960 

1679 

2008 

1856 

1634 

1737 

1644 

BF 

2713 

2543 

2872 

2707 

2566 

2683 

2563 

2439 

2418 

2355 

CFR 

1560 

1550 

3771 

3560 

3305 

3576 

3438 

3236 

3287 

3206 

CFL 

3672 

3212 

1517 

1440 

1417 

1470 

1472 

1462 

1413 

1472 

"BR 

885 

1095 

4972 

4681 

4258 

4871 

4703 

4373 

4626 

4523 

"BE 

3833 

3331 

1174 

1219 

1286 

1455 

1459 

1437 

1552 

1619 

CBR 

855 

1111 

4504 

4285 

3944 

4989 

4751 

4335 

5082 

4941 

CBL 

2736 

2524 

1803 

1768 

1775 

2448 

2302 

2126 

2755 

2706 

"BB- 

1509 

1511 

3306 

3181 

2960 

3979 

3827 

3505 

4300 

4247 

"Of- 

2644 

2610 

2993 

2920 

2907 

2944 

2948 

2944 

2883 

2944 

DB 

2203 

2463 

3652 

3623 

3634 

4291 

4165 

3989 

4564 

4535 

120 


WFL30F 

W8L15C 

"WBLi5""" 

WBL15F 

WBL30C 

W6L30 

WBL30F 

WFC 

~WF 

"WFF 

Null  lemp 

1629 

1747 

1254 

900 

1702 

1243 

845 

2455 

1951 

1512 

TC 

1780 

2428 

1947 

1603 

2403 

1952 

1562 

1526 

1106 

781 

F 

1504 

1926 

1445 

1101 

1901 

1450 

1060 

1844 

1384 

999 

FRC 

2351 

2659 

2166 

1812 

2614 

2155 

1757 

1503 

1207 

1010 

FR 

1856 

2046 

1553 

1199 

2001 

1542 

1144 

1796 

1386 

1075 

FLC 

836 

1338 

1145 

1041 

1793 

1534 

1336 

1616 

1428 

1207 

FL 

1137 

1373 

1012 

768 

1548 

1177 

867 

1987 

1617 

1270 

TC 

1032 

734 

453 

313 

957 

650 

468 

2260 

1932 

1611 

T~ 

1369 

1275 

844 

558 

1296 

883 

559 

2367 

1909 

1500 

WT~ 

2831 

2951 

2458 

2104 

2906 

2447 

2049 

2657 

2425 

2166 

TT~ 

1865 

1957 

1464 

1110 

1912 

1453 

1055 

2381 

1977 

1596 

"BC- 

2444 

1984 

1707 

1449 

1543 

1372 

1238 

3292 

2786 

2345 

"B- 

1689 

1689 

1232 

894 

1570 

1163 

813 

2539 

2031 

1590 

BRC 

2543 

2635 

2142 

1788 

2510 

2091 

1733 

3325 

2873 

2436 

BR 

1722 

1814 

1321 

967 

1749 

1300 

912 

2598 

2098 

1649 

BLC 

1962 

846 

599 

465 

555 

342 

240 

3188 

2682 

2241 

BL 

1559 

1343 

890 

584 

1258 

839 

509 

2553 

2045 

1604 

WRC 

3692 

3944 

3451 

3097 

3851 

3416 

3042 

2692 

2538 

2369 

WR 

3373 

3541 

3048 

2694 

3468 

3023 

2639 

2621 

2439 

2242 

WRF 

3026 

3188 

2695 

2341 

3123 

2674 

2286 

2498 

2286 

2059 

WFR15C 

3480 

3894 

3401 

3047 

3833 

3382 

2992 

2228 

2086 

1941 

WFR15 

3186 

3524 

3043 

2699 

3491 

3044 

2658 

2108 

1956 

1803 

WFR15F 

2967 

3207 

2726 

2380 

3146 

2701 

2315 

2111 

1919 

1732 

WFR30C 

3249 

3827 

3346 

3002 

3798 

3349 

2961 

1841 

1713 

1592 

WFR30 

3128 

3620 

3127 

2773 

3575 

3116 

2718 

1838 

1680 

1539 

WFR30F 

2820 

3230 

2737 

2383 

3185 

2726 

2328 

1752 

1558 

1391 

WBR15C 

3875 

4003 

3514 

3160 

3868 

3455 

3103 

3253 

3075 

2866 

WBR15 

3408 

3536 

3043 

2689 

3431 

3002 

2634 

3050 

2842 

2599 

WBR15F 

2961 

3087 

2594 

2240 

2998 

2561 

2185 

2805 

2555 

2278 

WBR30C 

3901 

3899 

3450 

3104 

3662 

3313 

3023 

3767 

3543 

3270 

WBR30 

3360 

3388 

2923 

2577 

3207 

2826 

2504 

3466 

3196 

2881 

WBR30F 

2850 

2928 

2443 

2089 

2795 

2382 

2030 

3090 

2778 

2439 

WLC 

1012 

458 

467 

801 

682 

706 

2318 

2134 

1913 

WL 

976 

494 

381 

387 

837 

662 

628 

2300 

2086 

1843 

WLF 

965 

585 

382 

310 

858 

613 

501 

2227 

1951 

1666 

WFL15C 

882 

716 

603 

613 

1123 

940 

902 

1978 

1808 

1603 

WFL15 

808 

718 

585 

571 

1145 

942 

870 

1898 

1722 

1511 

WFL15F 

777 

741 

576 

526 

1136 

909 

797 

1869 

1663 

1428 

WFL30C 

829 

1065 

914 

878 

1478 

1261 

1163 

1493 

1331 

1146 

WFL30 

767 

1071 

902 

846 

1502 

1267 

1141 

1505 

1335 

1138 

WFL30F 

741 

1025 

852 

788 

1460 

1221 

1083 

1551 

1373 

1166 

WBL15C 

1286 

306 

335 

441 

501 

470 

570 

2686 

2470 

2225 

WBL15 

1230 

362 

265 

275 

497 

362 

362 

2638 

2342 

2039 

WBL15F 

1201 

537 

314 

222 

638 

397 

289 

2507 

2155 

1818 

WBL30C 

1865 

437 

482 

560 

246 

289 

445 

3227 

2889 

2562 

WBL30 

1678 

486 

381 

357 

313 

216 

240 

2992 

2594 

2227 

WBL30F 

1567 

611 

386 

276 

478 

267 

181 

2849 

2413 

2020 

WFC 

1672 

2480 

2251 

2117 

2875 

2592 

2370 

704 

662 

531 1 

W7F- 

1566 

2314 

2049 

1883 

2617 

2314 

2070 

762 

638 

563| 

WFF 

1470 

2198 

1873 

1659 

2433 

2086 

1800 

880 

678 

535 

TF- 

2241 

2983 

2808 

2726 

3452 

3207 

3035 

1009 

1147 

-I223 

CFR 

3046 

3782 

3493 

3307 

4021 

3714 

3466 

1678 

1746 

1747 

CFL 

1514 

1580 

1457 

1461 

1965 

1776 

1736 

1258 

1200 

1163 

"TTT- 

4309 

4783 

4334 

3988 

4550 

4199 

3907 

2887 

2761 

2630 

TE- 

1631 

745 

914 

1094 

776 

901 

1141 

2517 

2353 

2164 

CBft 

4667 

3787 

3616 

3442 

3200 

3153 

3157 

4185 

3999 

3788 

CBL 

2596 

1190 

1377 

1533 

989 

1204 

1530 

3976 

3764 

3509 

TJB- 

4085 

2489 

2408 

2342 

1818 

1859 

2009 

5123 

4747 

4340 

T5F- 

2990 

3064 

2953 

2967 

3453 

3280 

3256 

2056 

2244 

2329 

DB 

4443 

3013 

3216 

3380 

2782 

3019 

3365 

4405 

4495 

4468 

121 


CF 

TFE |TFE 

TE 

TI TIE 

TEE 

TE 

TF 

TE- 

Nulllemp 

3481 

3463 

ET7E- 

3163 

3T3I 

— 3M 

3066 

2996 

4758 

4909 

”FC 

2544 

2588 

2280 

2516 

3029 

3717 

3765 

3775 

3861 

5326 

F 

2868 

2912 

2570 

2760 

2867 

3383 

3263 

3273 

4185 

5028 

FRC 

2097 

2025 

2531 

1785 

3754 

2622 

3978 

3656 

3320 

4273 

TFT- 

2696 

2706 

2658 

2430 

3321 

2871 

3383 

3289 

4001 

4550 

FlT 

2448 

3020 

2138 

3664 

2187 

4031 

2675 

3837 

3765 

4500 

”FE 

2931 

3195 

2621 

3293 

2600 

3394 

2710 

3200 

4248 

4563 

TC 

3076 

3524 

2158 

3728 

2043 

3459 

2071 

2785 

3785 

3916 

T~ 

3319 

3431 

2751 

3257 

2672 

3134 

2594 

2730 

4330 

4429 

wr~ 

2633 

1871 

3813 

1511 

4338 

1662 

4000 

2778 

3166 

3377 

"E- 

2981 

2663 

3267 

2329 

3386 

2350 

3168 

2626 

3958 

4069 

"EC 

4282 

4012 

3932 

2706 

3005 

1843 

1695 

1625 

5223 

3394 

"B- 

3565 

3517 

3247 

2973 

3042 

2818 

2666 

2620 

4794 

4505 

BRC 

4153 

3003 

4111 

1889 

3984 

1920 

2656 

1778 

4258 

3599 

"EE 

3594 

3266 

3324 

2752 

3223 

2749 

2795 

2513 

4551 

4458 

BLC 

4088 

4196 

2920 

3852 

2105 

2859 

1927 

1899 

4499 

3770 

"BI- 

3537 

3607 

2915 

3297 

2752 

3080 

2652 

2630 

4542 

4527 

WRC 

2560 

1264 

3934 

476 

5075 

1229 

4279 

2693 

2535 

2864 

WR 

2543 

1403 

3833 

773 

4818 

1314 

4142 

2668 

2684 

2971 

WRF 

2486 

1500 

3668 

954 

4485 

1391 

3935 

2593 

2785 

3064 

WFR15C 

2114 

1050 

3470 

622 

4817 

1687 

4651 

3193 

2337 

3326 

WFR15 

2040 

1152 

3322 

812 

4569 

1697 

4427 

3149 

2421 

3298 

WFR15F 

2111 

1341 

3297 

993 

4342 

1664 

4196 

2998 

2630 

3321 

WFR30C 

1755 

995 

3065 

793 

4522 

2104 

4830 

3624 

2266 

3695 

WFR30 

1808 

1126 

3054 

904 

4445 

2067 

4705 

3547 

2421 

3714 

WFR30F 

1824 

1314 

2926 

1078 

4177 

2055 

4387 

3413 

2609 

3710 

WBR15C 

3119 

1671 

4473 

533 

5266 

930 

3918 

2272 

2926 

2571 

WBR15 

2978 

1686 

4230 

754 

4847 

1041 

3751 

2213 

2951 

2710 

WBR15F 

2871 

1733 

3919 

953 

4418 

1194 

3522 

2148 

3006 

2875 

WBR30C 

3679 

2167 

4905 

711 

5114 

652 

3332 

1722 

3408 

2289 

W8R30 

3524 

2126 

4538 

890 

4671 

871 

3159 

1665 

3375 

2528 

WBR30F 

3326 

2128 

4104 

1086 

4261 

1135 

3019 

1741 

3389 

2812 

WLC 

2998 

3736 

1748 

4360 

1381 

3925 

1697 

2955 

3375 

3520 

'W~ 

3008 

3690 

1846 

4222 

1577 

3819 

1775 

2933 

3473 

3602 

WLF 

2993 

3547 

1953 

3903 

1766 

3586 

1894 

2806 

3580 

3731 

WFL15C 

2670 

3396 

1518 

4090 

1457 

4135 

2025 

3279 

3143 

3830 

WFL15 

2614 

3316 

1618 

3982 

1583 

4015 

2041 

3287 

3243 

3846 

WFL15F 

2625 

3259 

1779 

3837 

1702 

3828 

2064 

3206 

3406 

3891 

WFL30C 

2235 

2897 

1361 

3619 

1580 

4162 

2388 

3580 

2978 

4123 

WFL30 

2265 

2909 

1505 

3613 

1680 

4106 

2408 

3594 

3126 

4179 

WFL30F 

2323 

2955 

1637 

3629 

1752 

4010 

2362 

3544 

3260 

4151 

WBL15C 

3374 

4074 

2054 

4556 

1305 

3627 

1399 

2567 

3681 

3224 

WBL15 

3388 

3950 

2106 

4244 

1499 

3463 

1545 

2419 

3749 

3426 

WBI15F 

3309 

3743 

2177 

3867 

1802 

3290 

1790 

2418 

3804 

3639 

WBL30C 

3981 

4433 

2625 

4349 

1372 

2886 

1074 

1782 

4220 

2893 

WBL30 

3800 

4150 

2506 

4016 

1573 

2885 

1389 

1843 

4117 

3232 

WBL30F 

3697 

3983 

2503 

3821 

1848 

2956 

1712 

2014 

4130 

3571 

WFC 

1314 

1756 

1490 

2404 

2859 

3825 

3799 

4673 

2567 

4368 

"WF- 

1500 

1906 

1522 

2420 

2719 

3759 

3637 

4407 

2801 

4498 

WFF 

1704 

2028 

1632 

2398 

2691 

3665 

3535 

4141 

3015 

4620 

CT 

885 

1205 

1847 

1977 

3330 

3368 

4248 

4720 

1796 

3615 

CFR 

1460 

646 

2782 

780 

4179 

2191 

4397 

3663 

1573 

3018 

CFL 

1850 

2554 

916 

3304 

1919 

4229 

2889 

4069 

2279 

3874 

TE- 

2769 

1271 

4109 

21 

5288 

1138 

4262 

2656 

2514 

2709 

IT- 

3181 

3875 

1837 

4516 

928 

3638 

1362 

2602 

3424 

3085 

CBR 

4013 

2583 

4971 

889 

4612 

35 

2578 

1062 

3444 

1249 

Cbl 

4586 

4576 

3344 

3792 

1835 

2137 

305 

1119 

4171 

1668 

"EE- 

5355 

4133 

4843 

2441 

3350 

766 

1262 

116 

4226 

1437 

DF 

1626 

1238 

2064 

1786 

3375 

2819 

3775 

3607 

933 

2402 

TE~" 

3945 

3123 

4237 

2183 

3592 

842 

1700 

1326 

2710 

323 

APPENDIX  D 

TABLE  OF  MODIFIED  EUCLIDEAN  DISTANCE  OF  EACH  SCENARIO 
RESPECT  TO  THE  NEW  DATABASE  OF  SCENARIO  TEMPLATES  WITH 
TWO  SENSOR  MALFUNCTIONING 


NullTemp 

TC 

T 

TRC — 

TR 

TEC 

TE 

LC 

T 

m — 

Nulllemp 

10 

710 

151 

931 

215 

746 

87 

464 

74 

1271 

TC- 

743 

! 9 

194 

750 

554 

1063 

718 

1167 

799 

1972 

F 

258 

166 

13 

697 

261 

778 

287 

712 

322 

1487 

FRC 

971 

747 

716 

8 

246 

1655 

1046 

1455 

1043 

1052 

TR 

310 

556 

263 

243 

1 9 

1010 

379 

764 

374 

1063 

FLC 

810 

966 

697 

1525 

949 

22 

319 

544 

682 

2039 

TE 

165 

707 

234 

1050 

364 

287 

10 

319 

149 

1426 

TC- 

520 

1190 

661 

1441 

725 

386 

267 

U 

224 

1781 

L 

81 

775 

222 

982 

280 

635 

92 

207 

1290 

"RC- 

1162 

1834 

1303 

943 

863 

1954 

1261 

1646 

1234 

TT 

"R- 

228 

916 

369 

665 

223 

964 

305 

682 

292 

459 

~wr~ 

706 

1408 

847 

1627 

911 

1498 

805 

1190 

778 

1967 

B 

28 

730 

169 

949 

233 

820 

127 

512 

100 

1289 

BRC 

893 

1593 

1034 

1770 

1074 

1685 

992 

1377 

965 

1324 

TsR- 

81 

783 

222 

1002 

286 

873 

180 

565 

153 

1076 

6LC 

707 

1407 

848 

1628 

912 

1401 

784 

447 

435 

1968 

BL 

75 

777 

216 

996 

280 

851 

174 

287 

19 

1336 

WRC 

2102 

2184 

1989 

1061 

1385 

2840 

2189 

2586 

2174 

473 

WR 

1824 

1988 

1749 

881 

1155 

2580 

1915 

2308 

1896 

303 

WRF 

1442 

1758 

1433 

745 

895 

2216 

1537 

1926 

1514 

151 

WFR15C 

2140 

1810 

1827 

859 

1323 

2770 

2203 

2624 

2212 

549 

WFR15 

1879 

1627 

1604 

652 

1086 

2471 

1924 

2333 

1943 

440 

WFR15F 

1486 

1446 

1311 

487 

797 

2188 

1565 

1970 

1558 

315 

WFR30C 

2175 

1407 

1642 

694 

1292 

2605 

2184 

2629 

2239 

798 

WFR30 

1920 

1324 

1473 

537 

1071 

2404 

1941 

2374 

1984 

671 

WFR30F 

1523 

1091 

1152 

332 

758 

2099 

1574 

2007 

1595 

600 

WBR15C 

2251 

2597 

2256 

1488 

1672 

3025 

2346 

2735 

2323 

616 

WBR15 

1717 

2217 

1788 

1244 

1284 

2509 

1816 

2201 

1789 

306 

WBR15F 

1344 

1914 

1443 

1083 

1015 

2080 

1421 

1798 

1408 

177 

WBR30C 

2135 

2741 

2248 

1876 

1816 

2927 

2234 

2619 

2207 

712 

WBR30 

1714 

2360 

1841 

1669 

1501 

2506 

1813 

2198 

1786 

529 

WBR30F 

1160 

1812 

1287 

1327 

1043 

1952 

1259 

1644 

1232 

393 

WLC 

1413 

1753 

1396 

2220 

1584 

593 

916 

301 

893 

2674 

~W~ 

1144 

1626 

1199 

1987 

1321 

494 

703 

160 

672 

2405 

WLF 

820 

1390 

913 

1717 

1021 

418 

471 

58 

428 

2081 

WFL15C 

1381 

1365 

1182 

1998 

1486 

495 

878 

367 

901 

2558 

WFL15 

1132 

1290 

1019 

1891 

1295 

308 

641 

254 

716 

2393 

WFL15F 

924 

1222 

883 

1719 

1093 

216 

467 

178 

564 

2185 

WFL30C 

1382 

1112 

1059 

1887 

1441 

378 

841 

508 

974 

2611 

WFL30 

1199 

977 

900 

1760 

1284 

223 

662 

495 

871 

2460 

WFL30F 

970 

956 

771 

1651 

1105 

110 

467 

398 

698 

2231 

WBL15C 

1511 

2071 

1604 

2408 

1712 

817 

1052 

395 

983 

2772 

WBL15 

1106 

1724 

1223 

2015 

1309 

768 

783 

204 

642 

2367 

WBL15F 

767 

1395 

884 

1676 

970 

713 

554 

83 

375 

2028 

WRL30C 

1464 

2132 

1605 

2385 

1669 

1254 

1189 

588 

1000 

2725 

WBL30 

1036 

1718 

1177 

1957 

1241 

1220 

915 

404 

644 

2297 

WBL30F 

709 

1399 

850 

1630 

914 

1121 

676 

265 

381 

1970 

WFC 

1861 

871 

1232 

864 

1162 

987 

1370 

1655 

1749 

2266 

"WF 

1510 

568 

895 

703 

899 

916 

1129 

1454 

1438 

2135 

WFF 

1231 

381 

654 

630 

722 

823 

920 

1265 

1183 

2004 

TF- 

2653 

1651 

2024 

1184 

1780 

1607 

2096 

2289 

2485 

1888 

CFR 

2834 

1852 

2205 

1325 

1949 

2352 

2497 

2838 

2778 

1281 

CFL 

2372 

1416 

1757 

1649 

1797 

1284 

1815 

1294 

1868 

3133 

TR 

3062 

2342 

2553 

1613 

2187 

3566 

3097 

3546 

3134 

1433 

~CU~ 

2366 

1972 

1981 

2785 

2393 

1366 

1839 

1250 

1838 

3543 

CBR 

3122 

3480 

3137 

2329 

2523 

3900 

3221 

3368 

3082 

1563 

CBL 

2627 

3247 

2754 

3506 

2810 

2029 

2198 

1609 

2139 

3622 

TB- 

2984 

3766 

3163 

3699 

3135 

3744 

3087 

2618 

2656 

2619 

T5F-" 

3825 

2823 

3196 

2316 

2940 

2739 

3268 

2781 

3337 

2234 

T5B- 

TT 

CD 

xF 

4871 

4618 

3794 

4018 

4023 

4208 

3589 

4137 

3044 

123 
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"R 

"BB 

"B 

1RB — 

"BR 

"BEB 

"BE 

WRB 

"WR- 

"WRF — 

Null  lemp 

186 

744 

46 

940 

97 

754 

78 

2319 

1972 

1439 

TB- 

907 

1477 

779 

1673 

830 

1487 

811 

2376 

2231 

1864 

F 

424 

992 

294 

1188 

345 

1002 

326 

2211 

1970 

1523 

FRC 

739 

1705 

1007 

1879 

1036 

1715 

1039 

1164 

1023 

804 

TR- 

316 

1044 

346 

1234 

391 

1054 

378 

1599 

1362 

991 

FLC 

928 

1544 

846 

1740 

897 

1554 

878 

2979 

2682 

2183 

TT- 

321 

899 

201 

1095 

252 

909 

233 

2456 

2121 

1594 

"EB 

666 

1254 

556 

1450 

607 

604 

348 

2829 

2482 

1949 

T 

231 

815 

117 

1009 

166 

517 

37 

2336 

1989 

1460 

RB- 

422 

1896 

1198 

1252 

949 

1906 

1230 

549 

304 

101 

"R- 

12 

962 

264 

828 

195 

972 

296 

1303 

996 

599 

"EB- 

&84 

8 

426 

854 

555 

1030 

730 

2505 

2294 

1931 

"B- 

206 

532 

12 

782 

63 

732 

92 

2221 

1906 

1409 

BRC 

725 

877 

753 

13 

432 

1633 

961 

1474 

1381 

1192 

"ER- 

147 

645 

77 

493 

10 

825 

149 

1886 

1607 

1170 

BLC 

883 

1089 

679 

1637 

794 

15 

375 

3016 

2669 

2136 

"BE 

253 

765 

103 

1005 

162 

403 

7 

2384 

2037 

1504 

WEB 

1180 

2394 

2034 

1336 

1633 

2846 

2176 

\1 

30 

131 

WR 

944 

2286 

1796 

1356 

1443 

2568 

1892 

69 

20 

49 

WRF 

650 

2040 

1446 

1276 

1153 

2186 

1510 

209 

84 

“T5 

WFR15C 

1234 

2670 

2128 

1712 

1769 

2884 

2208 

131 

140 

221 

WFR15 

1035 

2545 

1899 

1795 

1612 

2623 

1947 

230 

179 

186 

WFR15F 

748 

2220 

1522 

1664 

1301 

2230 

1554 

333 

218 

139 

WFR30C 

1359 

2909 

2211 

2305 

1972 

2919 

2243 

528 

509 

522 

WFR30 

1156 

2654 

1956 

2140 

1747 

2664 

1988 

483 

436 

415 

"WFR30F 

891 

2257 

1559 

1967 

1424 

2267 

1591 

580 

487 

392 

WBR15C 

1323 

2199 

2103 

1097 

1668 

2987 

2319 

114 

145 

272 

WBR15 

871 

1941 

1633 

913 

1240 

2461 

1785 

156 

97 

108 

WBR15F 

588 

1704 

1292 

800 

947 

2088 

1412 

267 

148 

75 

WBR30C 

1277 

1765 

1915 

833 

1506 

2847 

2203 

396 

389 

468 

WBR30 

942 

1626 

1558 

606 

1149 

2446 

1782 

401 

352 

355 

WBR30F 

546 

1278 

1052 

406 

707 

1896 

1228 

499 

392 

295 

WEB- 

1539 

1993 

1421 

2343 

1500 

877 

1069 

3652 

3337 

2824 

"WL- 

1274 

1812 

1168 

2074 

1231 

824 

848 

3401 

3074 

2555 

WLF 

958 

1510 

848 

1750 

907 

664 

576 

3117 

2778 

2249 

WFL15C 

1479 

2071 

1409 

2309 

1466 

1093 

1089 

3444 

3167 

2686 

WFL15 

1260 

1844 

1164 

2062 

1219 

1052 

908 

3347 

3048 

2543 

WFL15F 

1054 

1658 

960 

1854 

1011 

1008 

752 

3157 

2846 

2335 

WFL30C 

1498 

2116 

1418 

2312 

1469 

1378 

1178 

3407 

3162 

2701 

WFL30 

1325 

1933 

1235 

2129 

1286 

1415 

1075 

3276 

3025 

2556 

WFL30F 

1098 

1704 

1006 

1900 

1057 

1318 

894 

3133 

2854 

2363 

WBL15C 

1639 

1937 

1491 

2441 

1598 

707 

1123 

3808 

3469 

2940 

WBL15 

1246 

1598 

1098 

2036 

1193 

442 

746 

3409 

3066 

2535 

WBL15F 

917 

1347 

775 

1697 

854 

319 

455 

3070 

2727 

2196 

WBL30C 

1608 

1534 

1372 

2374 

1541 

384 

1044 

3763 

3416 

2893 

WBL30 

1194 

1266 

976 

1954 

1117 

136 

648 

3339 

2992 

2465 

WBL30F 

875 

1121 

685 

1635 

794 

65 

373 

3016 

2669 

2138 

WFC 

1703 

2595 

1897 

2779 

1936 

2605 

1929 

2292 

2215 

2002 

"WF- 

1442 

2244 

1546 

2434 

1591 

2254 

1578 

2229 

2140 

1895 

WFF 

1219 

1965 

1267 

2157 

1314 

1975 

1299 

2182 

2075 

1798 

CF 

2039 

3387 

2689 

3317 

2624 

3309 

2689 

1768 

1717 

1630 

CFR 

1912 

3398 

2830 

2492 

2489 

3578 

2902 

765 

826 

931 

CFL 

2338 

3062 

2400 

3300 

2457 

2040 

2064 

3245 

3154 

2895 

UR- 

2140 

2692 

2842 

1802 

2445 

3774 

3130 

371 

572 

851 

"CT- 

2464 

2236 

2230 

3216 

2421 

1238 

1942 

4245 

4024 

3605 

CBR 

2226 

1890 

2726 

1816 

2483 

2902 

3006 

913 

972 

1157 

CBL 

2645 

1387 

2235 

2245 

2312 

1543 

2235 

3882 

3747 

3494 

"BB 

2502 

1656 

2572 

1662 

2357 

1784 

2556 

2593 

2534 

2481 

"BF 

2867 

4289 

3801 

3397 

3454 

3505 

3529 

1676 

1749 

1870 

"BB- 

3681 

3241 

4213 

3287 

3986 

3477 

4229| 

2338 

2413 

2618 
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r WFR15C 

WFR15  IWFR15F 

WF&30C  | WFR30 

WFR30F 

WRR15C  | WBR15 

WBR15F 

WBR30C 

Nulllemp 

2243 

1964 

— MT 

2268 

1W 

1552 

2433 

TW 

1333 

2325 

TC- 

1860 

1761 

1512 

1573 

1364 

1191 

2914 

2444 

1948 

2966 

T 

1887 

1708 

1363 

1730 

1471 

1202 

2541 

2039 

1511 

2513 

FRC 

848 

677 

478 

755 

514 

321 

1750 

1422 

1148 

2034 

FR 

1411 

1184 

849 

1388 

1101 

796 

1969 

1545 

1127 

2077 

FLC 

2735 

2520 

2127 

2570 

2339 

2030 

3177 

2651 

2099 

3089 

TL 

2332 

2071 

1626 

2295 

2006 

1635 

2588 

2052 

1488 

2480 

TC 

2753 

2474 

2011 

2748 

2457 

2062 

2943 

2407 

1843 

2835 

T 

2260 

1981 

1522 

2279 

1966 

1575 

2450 

1916 

1358 

2344 

TH- 

581 

410 

247 

752 

657 

496 

687 

361 

169 

777 

"R- 

1269 

1032 

693 

1350 

1121 

824 

1427 

977 

569 

1401 

”BC 

2735 

2558 

2129 

2898 

2609 

2248 

2211 

1947 

1587 

1877 

B 

2213 

1958 

1503 

2272 

1957 

1570 

2251 

1771 

1251 

2109 

"S'RC 

1862 

1835 

1586 

2225 

2154 

1929 

1166 

976 

758 

976 

”BR 

1932 

1725 

1328 

2055 

1812 

1469 

1896 

1448 

976 

1768 

BLC 

2940 

2661 

2198 

2965 

2644 

2249 

3130 

2594 

2030 

2990 

"BE 

2308 

2029 

1566 

2335 

2012 

1617 

2498 

1962 

1398 

2386 

WRC 

147 

166 

205 

410 

451 

448 

205 

147 

225 

425 

WR 

131 

98 

95 

354 

359 

320 

321 

185 

189 

541 

WRF 

229 

136 

65 

416 

371 

278 

465 

243 

155 

649 

WFR15C 

1 5 

38 

115 

122 

171 

212 

563 

433 

451 

851 

WFR15 

50 

1 5 

40 

107 

104 

101 

716 

518 

468 

1010 

WFR15F 

161 

64 

218 

151 

80 

797 

537 

413 

1057 

WFR30C 

130 

131 

564 

■IS 

22 

85 

1170 

950 

880 

1504 

WFR30 

129 

96 

125 

42 

13 

32 

1091 

857 

765 

1413 

WFR30F 

244 

161 

114 

151 

54 

9 

1142 

870 

720 

1430 

WBR15C 

454 

495 

512 

827 

876 

857 

28 

52 

192 

140 

WBR15 

424 

399 

348 

753 

758 

685 

116 

20 

42 

206 

WBR15F 

475 

406 

303 

758 

729 

616 

255 

79 

13 

317 

WBR30C 

860 

889 

846 

1283 

1320 

1261 

74 

102 

214 

WBR30 

809 

810 

727 

1202 

1217 

1128 

133 

75 

101 

“ST 

W6R30F 

807 

760 

615 

1136 

1103 

964 

321 

161 

69 

259 

WLC 

3472 

3233 

2812 

3361 

3116 

2773 

3822 

3290 

2730 

3710 

"Wt~ 

3269 

3012 

2573 

3204 

2943 

2576 

3553 

3021 

2461 

3449 

WLF 

3009 

2742 

2291 

2968 

2693 

2314 

3243 

2707 

2143 

3131 

WFL15C 

3144 

2951 

2586 

2935 

2720 

2443 

3680 

3160 

2618 

3600 

WFL15 

3103 

2888 

2491 

2938 

2707 

2396 

3541 

3009 

2449 

3441 

WFL15F 

2961 

2728 

2313 

2840 

2595 

2260 

3333 

2801 

2241 

3235 

WFL30C 

3027 

2866 

2533 

2756 

2561 

2320 

3707 

3193 

2653 

3649 

WFL30 

2912 

2745 

2402 

2655 

2456 

2205 

3566 

3046 

2498 

3498 

WFL30F 

2833 

2640 

2267 

2624 

2407 

2124 

3365 

2837 

2281 

3277 

WBL15C 

3700 

3433 

2982 

3649 

3384 

3005 

3934 

3398 

2834 

3798 

WBL15 

3317 

3044 

2587 

3292 

3011 

2624 

3529 

2993 

2429 

3399 

WBL15F 

2978 

2705 

2248 

2963 

2672 

2285 

3190 

2654 

2090 

3068 

WBL30C 

3697 

3418 

2955 

3690 

3401 

3006 

3847 

3331 

2777 

3655 

WBL30 

3269 

2990 

2527 

3276 

2973 

2578 

3435 

2911 

2353 

3267 

WBL30F 

2942 

2663 

2200 

2957 

2646 

2251 

3124 

2592 

2030 

2982 

WFC 

1728 

1651 

1520 

1377 

1234 

1141 

2980 

2634 

2318 

3244 

"WF- 

1673 

1592 

1435 

1334 

1173 

1060 

2889 

2521 

2167 

3117 

WFF 

1650 

1559 

1374 

1333 

1158 

1023 

2804 

2412 

2024 

2990 

EJF- 

1234 

1187 

1176 

917 

864 

875 

2458 

2184 

2024 

2786 

CFR 

419 

510 

661 

306 

427 

564 

1329 

1191 

1201 

1641 

CFL 

2689 

2608 

2439 

2334 

2185 

2062 

3901 

3523 

3149 

4111 

HR 

437 

624 

799 

594 

721 

868 

511 

641 

869 

683 

HE 

3859 

3720 

3411 

3568 

3385 

3170 

4505 

4045 

3543 

4337 

CBft 

1349 

1400 

1409 

1750 

1785 

1758 

665 

805 

1013 

591 

CBL 

4232 

4131 

3774 

4441 

4282 

3977 

3486 

3326 

3068 

3094 

HR 

3129 

3140 

2967 

3572 

3569 

3414 

2067 

2029 

1977 

1675 

HF- 

1352 

1455 

1616 

1235 

1388 

1533 

2220 

2098 

2124 

2522 

HR 

2758 

2825 

2854 

3103 

3202 

3193 

2108 

2260 

2480 

2024 

126 


WBR30 

"WBR30T 

wnc — 

~WC 

W 

WFL15C 

"WFUi5 

■WFCT5T 

WFL30C 

WFL30 

Nulllemp 

1772 

1196 

1360 

1127 

772 

1310 

1107 

807 

1292 

1098 

PC 

2483 

1915 

1917 

1768 

1463 

1437 

1424 

1290 

1119 

1075 

f 

1998 

1430 

1536 

1351 

1020 

1232 

1139 

935 

1042 

934 

FRC 

1809 

1495 

2341 

2128 

1775 

2135 

2016 

1758 

1973 

1851 

TR- 

1692 

1234 

1648 

1423 

1072 

1538 

1371 

1087 

1456 

1294 

FLC 

2550 

1982 

756 

659 

564 

504 

383 

307 

362 

230 

TL~ 

1927 

1351 

937 

756 

507 

827 

648 

432 

773 

597 

~nr~ 

2282 

1706 

280 

167 

52 

320 

207 

87 

452 

348 

~r~ 

1797 

1231 

913 

720 

437 

905 

744 

504 

957 

805 

RC 

590 

446 

2568 

2331 

1966 

2518 

2315 

2009 

2500 

2306 

R 

1002 

612 

1578 

1345 

990 

1528 

1325 

1025 

1510 

1316 

“BC- 

1550 

1212 

1924 

1753 

1432 

2018 

1837 

1531 

2022 

1850 

1594 

1066 

1416 

1185 

824 

1380 

1179 

873 

1364 

1172 

BRC 

649 

379 

2297 

2060 

1695 

2249 

2046 

1740 

2231 

2037 

"BR 

1273 

775 

1487 

1250 

885 

1437 

1234 

928 

1419 

1225 

BLC 

2469 

1893 

783 

730 

565 

1019 

974 

800 

1243 

1207 

BL 

1837 

1261 

1053 

860 

551 

1071 

920 

662 

1137 

995 

WRC 

452 

516 

3490 

3265 

2906 

3354 

3201 

2919 

3248 

3098 

WR 

516 

514 

3218 

2989 

2628 

3104 

2941 

2651 

3014 

2856 

WRF 

550 

470 

2842 

2609 

2246 

2754 

2577 

2279 

2688 

2518 

WFR15C 

848 

838 

3492 

3291 

2944 

3228 

3131 

2897 

3034 

2928 

WFR15 

963 

897 

3181 

2980 

2641 

2939 

2832 

2596 

2761 

2647 

WFR15F 

948 

820 

2862 

2645 

2290 

2684 

2549 

2283 

2550 

2414 

WFR30C 

1451 

1363 

3423 

3258 

2937 

2995 

2966 

2802 

2697 

2643 

WPR30 

1342 

1236 

3186 

3009 

2682 

2820 

2765 

2577 

2562 

2488 

WFR30F 

1309 

1155 

2857 

2668 

2327 

2535 

2460 

2250 

2309 

2219 

WBR15C 

201 

349 

3647 

3414 

3051 

3565 

3386 

3088 

3503 

3331 

WBR15 

163 

201 

3123 

2886 

2521 

3063 

2870 

2564 

3025 

2841 

WBR15F 

202 

156 

2694 

2461 

2106 

2638 

2441 

2141 

2608 

2420 

WBR30C 

49 

201 

3525 

3288 

2923 

3487 

3288 

2982 

3461 

3271 

WBR30 

18 

72 

3114 

2877 

2512 

3068 

2867 

2561 

3046 

2854 

WBR30F 

94 

18 

2562 

2325 

1960 

2514 

2313 

2007 

2492 

2300 

WLC 

3175 

2599 

29 

58 

137 

71 

76 

118 

231 

233 

WL 

2906 

2330 

46 

23 

48 

96 

57 

51 

264 

228 

WLF 

2582 

2006 

116 

85 

16 

160 

91 

35 

312 

248 

WFL15C 

3075 

2517 

181 

196 

247 

60 

140 

61 

85 

WFL15 

2894 

2318 

164 

145 

162 

30 

19 

51 

70 

48 

WFL15F 

2686 

2110 

194 

141 

120 

92 

21 

136 

76 

WFL30C 

3122 

2554 

380 

383 

414 

84 

127 

2l5 

34 

WFL30 

2961 

2385 

453 

432 

431 

147 

152 

202 

41 

""23 

WFL30F 

2732 

2156 

448 

399 

364 

188 

147 
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